AIR  58600- 11/78-TR 


to  Performance  Assessment^ 


E An  Elaboration  and  Refinement 
^ of  an  Information  Processing 
n.  Fterformance  Battery. 


Ted  W.  Allen 
Andrew  M.^tose 
Leslie  J. /Kramer 


Prepared  for  the  Personnel  and  Training  Research  Programs,  Psychological  Sciences  Division, 
— l Office  of  Naval  Research,  Arlington,  Virginia. 


Contract  No. 


TECHI 


AMERICAN  INSTITUTES  FOR  RESEARCH/1066  Thomas  Jefferson  Street,  NW,  Washington,  DC  20007 


Approved  for  public  release;  distribution  unlimited,  Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose 

of  the  United  States  government. 


rnmmmmm 


CURITY  CLASSIFICATl 


«TT«T.*iTT?IZT^i;/,.fLl»7n-CTr77: 


REPORT  DOCUMENTATION  PAGE 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


CATALOG  NUMBER 


Technical  Report  No.  3 


4.  TITLE  land  Subtitle) 

"An  Information  Processing  Approach  to 
Performance  Assessment:  III.  An 
Elaboration  and  Refinement  of  an  Informa- 
tion Processing  Performance  Battery" 


5.  TYPE  OF  R 

Technical  Report 
16  Nov.  1977  - 30  Nov.  19' 


6.  PERFORMING  org  report  NUMBER 
AIR  58  500-TR  ^ 


7.  AUTHOR  Itl 

T.  W.  Allen 
A.  M.  Rose 
L.  Kramer 


9.  PERFORMING  ORGANIZATION  NAME  AND  AOORESS  y 


8.  CONTRACT  OR  GRANT  NUMBERIll 

Hftnfti  ^ r .mi  ^ 


N00014-76-C-0871 1 


American  Institutes  for  Research 
1055  Thomas  Jefferson  St. , N.W. 

Washington,  D.C.  20007  


1 1 . CONTROLLING  OFFICE  NAME  ANO  AOORESS 
Personnel  and  Training  Research  Programs, 
Office  of  Naval  Research  - Code  458 
Arlington,  Virginia  22217 


14.  MONITORING  AGENCY  NAME  & AOORESS  (if  different  from  Controlling  Office) 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  & WORK  UNIT  NUMBERS 


61153N;  RR042-04-01; 

RR042-04;  NR  150-391 


12.  REPORT  DATE 

November  30,  1978 


13.  NUMBER  OF  PAGES 

125 


15.  SECURITY  CLASS,  lof  thi*  report) 


UNCLASSIFIED 


15a.  DECLASSIFICATION/DOWNGRADING 
SCHEDULE 


16.  DISTRIBUTION  STATEMENT  lof  this  Report) 


Approved  for  public  release?  distribution  unlimited. 


1 7.  DISTRIBUTION  STATEMENT  (of  the  abstract  entered  in  Block  20.  if  different  from  Report) 


18.  SUPPLEMENTARY  NOTES 


19  KEY  WOROS  (Continue  on  reverse  side  if  necessary  and  identify  by  block  numoeri 


Personnel  Technology  ___ 

Individual  Differences  r\  -t  O V / 

Personnel  Assessment  Q |5  | \J  ™ ‘ 

Information  Processing  • V/ 

Cognitive  Processing  


20  ABSTRACT  (Continue  on  side  if  necnury  and  identify  by  block  number » 
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battery  is  being  designed  to  possess  high  reliability  and  predictive 
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validity  for  a wide  variety  of  criterion  tasks.  Equally  important,  the 
battery  is  being  designed  to  include  tests  that  possess  construct  valid- 
ity: there  will  be  a firm  theoretical  and  empirical  base  for  inferring 
the  information  processing  structures  that  the  tests  purport  to  measure. 

The  major  purpose  of  the  present  study  was  to  determine  the  prop- 
erties of  a set  of  tasks  selected  for  inclusion  in  the  test  battery. 
Three  questions  were  of  primary  interest:  the  replicability  of  previous 
findings  with  alternate  forms  of  the  tasks,  the  adequacy  of  the  tasks 
to  provide  measures  of  individual  differences,  and  the  adequacy  of  the 
tasks  to  provide  measures  of  information  processing  operations. 

Sixty-eight  subjects  were  tested  twice  on  each  task.  The  tasks 
investigated  included: 

Physical  Match 
Letter  Rotation 
Scan  and  Search 
Set  Membership 
Letter  Recall 
Mental  Addition 
Sentence  Recall 
Sentence  Recognition 

In  general,  the  results  showed  the  forms  of  tasks  used  here  to  be 
quite  compatible  with  previous  findings  for  all  tasks  but  one.  Sentence 
Recognition.  Even  for  this  task,  there  is  support  in  the  experimental 
literature  for  the  obtained  findings.  The  support  for  individual 
difference  measures  and  measures  of  information-processing  operations 
varied  from  task  to  task.  The  summary  presents  our  recommendations 
regarding  the  inclusion  of  various  tasks  and  measures  in  the  battery. 
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INTRODUCTION 


The  Information  Processing  Performance  Battery  (IPPB)  is 
an  attempt  to  apply  the  theories  and  methods  of  laboratory- 
based  studies  of  human  cognitive  performance  to  the  area  of 
performance  assessment.  The  primary  rationale  developed  through 
an  ongoing  series  of  experiments  (Rose,  1974;  Rose  and  Fernandes, 
1977;  Fernandes  and  Rose,  1978)  is  that  individuals  can  poten- 
tially be  characterized  in  terms  of  parameters  derived  from 
models  of  selected  information  processing  tasks.  If  these 
parameters  can  be  demonstrated  to  meet  standard  test-item  cri- 
teria, then  a test  battery  comprised  of  such  measures  would 
not  only  be  potentially  predictive  of  performance  on  a wide 
variety  of  real-world  tasks  but  would  also  be  firmly  based  in 
theory.  Such  a test  battery  would  represent  a significant 
advance  over  standard  selection  and  placement  tests;  it  would 
promote  increased  understanding  of  the  cognitive  operations 
involved  in  any  criterion  task  shown  to  be  related  to  constructs 
in  the  test  battery. 

The  basic  approach  involved  in  this  research  is  exemplified 
by  Rose  (1974) . The  strategy  was  to  select  experimental  tasks 
from  the  psychological  literature  that  had  been  demonstrated 
to  be  valid  measures  of  information  processing  constructs.  A 
set  of  tasks  was  selected;  each  task  was  adapted  to  fit  logistic 
demands  of  time  and  equipment,  and  then  administered  to  a large 
group  of  subjects.  Extensive  correlational  analyses  were  con- 
ducted to  determine  the  relationships  among  the  tasks  and  the 
individual  task  reliabilities.  These  procedures  resulted  in  a 
set  of  tasks  which  were  reliable,  statistically  independent, 
and  (arguably)  possessing  high  construct  validity. 

The  Rose  and  Fernandes  (1977)  study  extended  this  approach 
by  hypothesizing  a set  of  constructs  ("operations")  which  were 
used  to  model  performance  for  each  task.  Since  most  of  the 


task  parameters  could  be  cast  as  time  measures,  it  was  possible 
to  employ  regression  techniques  to  "converge"  upon  these  opera- 
tions. Some  fairly  simple  assumptions,  such  as  linear  additive 
stages  and  independence  of  operations,  led  to  the  estimation 
of  durations  for  some  of  these  operations.  More  importantly, 
the  generation  of  these  constructs  provided  a valuable  heuristic 
device  for  the  interpretation  of  task  performance  and  provided 
an  initial  empirical  basis  for  the  isolation  of  basic  informa- 
tion processing  components. 

Fernandes  and  Rose  (1978)  attempted  to  extend  the  methodology 
to  the  realm  of  memorial  tasks.  Based  on  a study  by  Underwood, 
Boruch,  and  Malmi  (1977) , several  memory- related  tasks  were 
selected  for  more  detailed  evaluation.  Although  modeling 
of  these  tasks  in  terms  of  operations  was  not  attempted,  it  was 
possible  to  examine  the  obtained  relationships  among  the  tasks 
for  commonalities  that  could  be  interpreted  in  information  pro- 
cessing terms. 

The  present  study  is  similar  to  the  above  studies  in  that 
the  general  approach  was  the  same:  the  literature  was  reviewed 
in  order  to  select  candidate  paradigms,  these  paradigms  were 
adapted  to  meet  logistic  limitations,  and  the  tasks  were  admin- 
istered twice  to  subjects.  The  major  differences  between  this 
study  and  the  others  are  that  first,  several  tasks  previously 
included  in  the  IPPB  were  readministered,  primarily  to  test 
for  alternate-form  consistencies  and  to  capitalize  on  the  pre- 
vious findings  for  interpretation  of  results.  Second,  a number 
of  "new"  treatments  were  built  into  this  study.  These  treat- 
ments extended  the  theoretical  underpinnings  of  the  selected 
tasks,  thus  allowing  for  "stronger"  interpretations  of  the 
phenomena  under  study.  Third,  this  study  made  greater  use  of 
analysis  of  variance  techniques  for  the  isolation  of  potential 
individual  difference  parameters.  These  changes  in  analytical 
procedures  are  discussed  below. 
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General  Task  and  Operations  Overview 

The  task  selection  process  was  based  upon  an  extension  of 
the  approach  advocated  by  Rose  and  Fernandes  (1977).  They 
suggest  that  information  processing  tasks  can  be  described  as 
requiring  some  combination  of  information  manipulation  opera- 
tions, where  an  operation  represents  a set  of  control  processes 
that  must  be  performed  in  order  to  successfully  complete  the 
task  (see  Carroll,  1974) . Rose  and  Fernandes  arrived  at  eight 
specific  operations  necessary  to  describe  the  tasks  used  in 
their  study.  In  the  present  study  we  have  refined  their  list 
of  operations,  limiting  the  scope  of  some  operations,  more 
clearly  delineating  the  boundaries  of  others,  and  adding  new 
operations  in  order  to  cover  a wider  range.  These  operations 
are  described  below. 

Encoding : The  operation  by  which  information  is  input 

into  the  information  processing  system.  The  encoding  processes 
are  primarily  concerned  with  making  the  preliminary  analysis 
of  external  information  necessary  to  get  that  information  into 
the  system.  Some  processes  that  might  occur  in  the  context 
of  this  operation  are:  attribute  or  feature  extraction,  selec- 
tion, erasure,  masking,  and  temporal  summation. 


Abstraction:  The  operation  by  which  new  structures  are 

generated  from  information  already  in  the  system.  The  abstrac- 
tion processes  are  primarily  concerned  with  generation  of  higher 
level  structures  beyond  those  initially  encoded  in  the  system. 
Processes  that  might  occur  in  this  operation  include:  Neisser's 
(1967)  synthesis,  schema  formation,  concept  formation,  and 
categorization. 


Transformation : The  operation  by  which  information  is 
converted  into  an  equivalent  structure  that  is  required  for 
successful  task  performance.  In  contrast  to  abstraction,  the 
processes  involved  in  transformation  do  not  involve  the  genera- 
tion of  new  (unencoded)  information  but  rather  require  the 
application  of  some  stored  rules  to  the  information  structure 
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already  present.  Some  processes  that  could  occur  in  a trans- 
formation o eration  are:  rotation,  chunking,  chaining,  updating, 
and  segmentation. 
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Recoding : The  operation  by  which  the  information  is  con- 

verted into  a corresponding  structure  of  different  form  or 
composition.  In  contrast  to  transformation,  the  recoding  pro- 
cesses require  the  generation  of  new  information  that  is  not  an 
end-product  of  the  encoding  operation.  In  contrast  to  abstrac- 
tion, the  recoding  operation  does  not  require  a "synthesis" 
or  "formation"  so  much  as  a substitution  of  corresponding 
structures  of  different  types.  Recoding  processes  would  include, 
for  example,  changing  the  structure  of  information  in  the  sys- 
tem from  a verbal  form  to  an  imaginal  form,  or  from  acoustic 
to  semantic. 

Storage:  The  operation  by  which  new  information  is  inte- 

grated into  the  existing  organization  of  information  structures 
and  the  processes  impinging  on  the  information  while  there. 
Storage  processes  might  include:  decay,  degradation,  displace- 
ment, tagging,  unlearning,  and  Estes'  (1972)  reverberation. 

Retrieval:  The  operation  by  which  previously  stored 

information  is  made  available  to  the  processing  system.  Whether 
it  is  indicative  of  the  extent  of  research  on  the  operation  or 
the  primitive  nature  of  the  operation,  the  literature  contains 
no  exemplary  processes  for  the  retrieval  operation. 


Searching : The  operation  by  which  an  information  structure 

is  examined  for  the  presence  or  absence  of  one  or  more  properties. 
The  information  structure  examined  may  be  one  already  in  the 
processing  system  or  one  external  to  it  (e.g.,  a visual  array). 

The  searching  operation  might  include  processes  such  as  attri- 
bute verification  and  scanning. 

Comparison:  The  operation  by  which  two  entities,  either 

internal  or  external  to  the  processing  system,  are  judged  to  be 
the  same  or  different.  An  example  of  the  processes  involved  in 
the  comparison  operation  would  be  a matching  process. 
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Decision  and  Response:  The  operation  by  which  the  appro- 
priate motor  action  is  selected  and  executed.  Among  the  many 
processes  in  this  operation  are  monitoring  and  evaluation 
processes . 

The  operations  described  above  represent  only  a part  of 
the  information  necessary  to  completely  describe  task  perfor- 
mance. At  the  very  least,  one  might  need  information  about  the 
contents  or  form  of  the  information  structure  to  which  the 
operation  is  applied  (e.g.,  a comparison  operation  might  respond 
differently  to  pictorial,  acoustic  or  semantic  information)  and 
one  needs  information  about  the  state  of  system  stress  (i.e., 
the  amount  of  incoming  information  and  required  speed  of  pro- 
cessing) . For  the  tasks  described  below  we  shall  assume  that 
content  of  the  operations  is  irrelevant.  In  most  cases  the 
design  of  the  tasks  and  data  comparisons  support  this  assumption; 
however,  since  we  cannot  at  this  time  precisely  specify  the 
effects  of  contents  or  the  interaction  of  contents  and  operations 
we  have  a problem  of  " identif iability"  and  would  have  to  make 
this  or  an  equivalent  assumption  regardless  of  task  and  data 
comparison  considerations.  System  stress  can  also  be  assumed 
to  be  comparable  in  the  different  tasks.  The  possible  effects 
of  these  two  variables  will  be  entertained  where  relevant  in 
the  discussion. 

Listed  below  is  a description  of  the  eight  tasks  included 
in  the  present  study.  Each  description  is  accompanied  by  a 
discussion  of  the  operations  involved.  The  nature  of  their 
involvement  in  each  task  is  summarized  in  Table  1. 

Task  Descriptions 

Letter  Recall  Task.  The  letter  recall  task  is  a serial, 
short-term  retention  task  designed  to  embody  the  spirit  of  both 
the  memory  span  task  and  Waugh  and  Norman's  (1965)  probe-digit 
task.  These  two  tasks  are  used  to  investigate  the  limited 
(storage  or  processing)  capacity  of  the  memory  system.  In  a 
typical  memory  span  task,  subjects  study  lists  of  items  of 
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TABLE  1. 

Operational  Analysis  of  Tasks 


Information  Processing  Operations 


1 

Tasks 

Encoding 

Abstrac- 

tion 

Trans- 

forma- 

tion 

Recod- 

ing 

Storage 

Retrieval 

Search 

Com- 

parison 

Decision- 

response 

I 

1. 

Letter  recall 

minor 

minor 

minor 

minor 

major 

major 

minor 

minor 

minor 

2. 

Mental  addition 

minor 

minor 

major 

minor 

major 

major 

minor 

minor 

minor 

3. 

Sentence  recall 

minor 

major 

minor 

minor 

major 

major 

minor 

minor 

minor 

Ik 

4. 

Sentence  recognition 

minor 

major 

minor 

minor 

major 

minor 

major 

minor 

minor 

5. 

Letter  rotation 

minor 

minor 

major 

minor 

minor 

minor 

minor 

major 

minor 

n 

6. 

Physical  match 

minor 

minor 

minor 

minor 

minor 

minor 

minor 

major 

minor 

7. 

Set  membership 

minor 

minor 

minor 

minor 

major 

minor 

major 

minor 

minor 

8. 

Scan  and  search 

minor* 

minor 

minor 

minor 

major 

minor 

major 

minor 

minor 

major  = operation  is  of  MAJOR  importance  in  determining  task  performance, 
minor  * operation  is  of  MINOR  importance  in  determining  task  performance. 
* For  degraded  stimuli,  encoding  is  of  MAJOR  importance 
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increasing  length  which  must  be  recalled  in  order  from  a single 
presentation.  Typical  results  show  the  percentage  of  items 
recalled  to  remain  at  about  100%  for  the  first  4+1  items  and 
to  decrease  rapidly  thereafter.  Usually  the  memory  span  is 
defined  as  the  number  of  items  recalled  correctly  50%  of  the 
time  and  is  typically  7+2  items.  A better  measure  of  capacity 
however  might  be  the  amount  recalled  correctly  at  or  near  100% 
of  the  time  in  which  case  4+1  would  be  the  capacity  estimate. 

The  probe-digit  task  consists  of  the  presentation  of  a long 
series  of  items  (e.g. , 14  digits)  and  the  immediate  probing  of 
retention  by  re-presenting  one  of  the  items  from  the  series. 

The  subject  is  to  recall  the  item  which  immediately  followed 
the  probe  in  the  series.  The  task  typically  demonstrates  the 
displacement  effect,  that  later  items  in  the  series  tend  to 
displace  or  knock  out  earlier  items  from  the  limited  capacity 
system. 

In  the  present  task  the  subject  studies  lists  of  varying 
length  (5-10  letters)  and  must  recall  the  last  five  letters  in 
order.  Since  the  subject  does  not  know  when  the  list  will  end 
and  is  instructed  to  retain  only  the  last  five  items,  we  would 
expect,  theoretically  at  least,  the  subject  to  be  engaged  in 
a procedure  of  filling  the  system  to  near  capacity  (4+1  items) 
and  then  displacing  the  oldest  items  as  new  items  are  processed. 
Variance  in  task  performance  should  result  from  two  sources: 

(1)  the  size  of  the  individual's  memory  capacity  and  (2)  the 
individual's  ability  to  control  the  displacement  process.  Thus 
with  respect  to  the  operations  discussed  previously  we  expect 
the  storage  operations  and  retrieval  operations  to  be  the  major 
factors  in  task  performance. 

Procedure.  In  this  task  subjects  recalled  the  last  five 
letters  of  a series  of  letters.  In  each  tri  ! subjects  heard 
a series  of  letters  presented  at  the  rate  of  one  letter  per 
second.  At  the  end  of  the  series  subjects  heard  the  word 
"RECALL"  and  were  given  fifteen  seconds  to  write  down  the  last 


five  letters  of  the  series  in  the  order  in  which  they  occurred. 
Instructions  told  subjects  to  leave  a blank  space  if  they 
couldn't  remember  a letter,  and  if  they  could  remember  the 
letters  but  not  their  order,  they  were  to  guess  the  positions 
of  the  letters. 

The  following  shows  the  events  of  two  trials: 

STIMULI  RESPONSES 

"K  C M T N,  Recall"  K C M T N 

"A  M G S K P F C,  Recall"  S K P F C 

Stimuli  and  design.  This  task  was  presented  twice  a day. 
Each  administration  consisted  of  five  trials  explicitly  desig- 
nated as  practice  and  twenty-two  critical  trials  with  no  feed- 
back provided.  There  were  three  trials  for  each  series  of  5, 

6,  7,  8,  9,  and  10  letters.  These  18  trials  were  arranged 
randomly  with  respect  to  the  number  of  letters  in  each  trial 
with  the  restrictions  that  the  first  trial  be  a five-letter 
trial  and  no  consecutive  trials  have  stimuli  of  the  same 
length.  The  remaining  four  trials  were  non-scored  five-letter 
series.  These  occurred  as  the  second,  fourth,  sixth,  and 
twenty-second  trials  and  were  inserted  with  the  purpose  of 
discouraging  subjects  from  ignoring  the  first  few  letters  of 
any  trial. 

The  stimulus  for  each  trial  was  constructed  by  random 
selection  from  the  set  of  consonants,  excluding  W and  Y,  with 
the  constraints  that  no  letter  be  repeated  during  a trial  and 
a maximum  of  three  acoustically  similar  letters  be  permitted  to 
occur  consecutively  within  a trial. 

Variables.  Three  principal  measures  were  computed  for 
the  task.  The  order-recall  measure  consisted  of  the  number  of 
letters  recalled  from  the  last  five  letters  in  the  series  in 
the  correct  positions;  thus  credit  was  given  only  if  a letter 
recalled  was  one  of  the  last  five  letters  in  the  series  and  if 
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that  letter  was  recalled  in  its  correct  position.  The  derived - 
free-recall  consisted  of  the  number  of  letters  recalled  from 
the  last  five  letters  in  the  series  irrespective  of  the  corres- 
pondence of  their  positions.  Here  credit  was  given  for  any 
letter  recalled  from  the  last  five  letters,  regardless  of  the 
position  in  which  it  was  recalled.  Lastly,  the  series-recall 
measure  consisted  of  the  number  of  letters  recalled  from  any 
position  in  the  series.  For  each  principal  measure,  two  derived 
measures  were  obtained:  mean  performance  and  the  slope  of  the 
function  relating  amount  recalled  and  condition. 

Mental  Addition  Task.  The  mental  addition  task  is  a 
replication  of  a task  designed  by  Hitch  (1978)  to  study  pro- 
cessing capacity  in  the  memory  system.  Hitch  read  to  his  sub- 
jects two  numbers  (e.g.,  "four  hundred  eighty-seven  plus  twenty- 

i 

six")  which  they  had  to  add  together  mentally,  without  writing 
or  seeing  the  addends,  and  report  the  sum.  Separately  he 
varied : 

(1)  the  size  of  the  addends  (i.e.,  two  three-digit  numbers 
or  a three-digit  number  and  a two-digit  number) , 

(2)  the  amount  and  location  of  "carrying"  operations 
(i.e.,  no  carrying,  carrying  into  tens  position, 
carrying  into  hundreds  position,  and  carrying  into 
both  tens  and  hundreds  positions) . 

Hitch  found  that  the  time  to  complete  the  addition  was  greater 
for  the  two  three-digit  addends  than  for  the  three-digit  and 
two-digit  addends,  and  that  the  time  increased  with  the  number 
of  carrying  operations  required. 

The  mental  addition  task  in  the  present  study  combines  the 
two  above-mentioned  variables.  Theoretically,  variance  in  task 
performance  across  individuals  should  depend  on  three  factors: 

(1)  The  individual's  ability  to  store  and  maintain  the 
addends  and  carries, 

(2)  The  individual's  ability  to  rapidly  retrieve  both 
the  rules  of  addition  and  the  addends  and  carries, 
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(3)  The  individual's  ability  to  actually  perform  the 

addition;  in  our  terminology,  this  can  be  categorized 
as  a transformation  operation. 

Thus,  the  major  operations  involved  in  task  performance  should 
be  the  storage,  retrieval,  and  transformation  operations. 

Procedure.  Subjects  were  required  to  mentally  add  two 
multiple-digit  numbers.  Subjects  heard  a cue  ("READY")  followed 
by  two  numbers.  They  then  had  five  seconds  to  add  the  numbers 
before  they  heard  the  word  "STOP".  Subjects  were  instructed 
to  write  down  each  digit  of  the  answer  as  soon  as  the  addition 
for  that  digit  was  complete.  They  were  also  told  not  to  guess. 
Here  is  the  sequence  of  events  of  two  trials: 

STIMULI  RESPONSES 

"Ready,  two  hundred  fifty-eight  plus  three 
hundred  twenty-seven,  ...  (five  second 

pause)  ...»  STOP"  585 

"Ready,  four  hundred  ninety- two  plus  two 

hundred  forty-one,  ...  (five  second  pause)  ..., 

STOP"  7 3 3 

Stimuli  and  design.  This  task  was  administered  twice  each 
day  and  subjects  completed  twenty-four  trials  during  each 
administration.  They  received  no  feedback.  Fifty  percent  of 
the  trials  required  the  addition  of  two  3-digit  numbers;  in 
the  remaining  trials  subjects  added  a 2~digit  number  to  a 3- 
digit  number.  The  trials  were  divided  equally  with  respect  to 
the  number  of  carry-overs  required  to  complete  the  addition  so 
that  25%  of  the  trials  required  no  carries,  25%  required  a 
carry  from  the  one's  column  to  the  ten's  column,  25%  required 
a carry  from  the  ten's  column  to  the  hundred's  column,  and  25% 
required  carries  from  both  the  one's  and  the  ten's  columns  to 
the  ten's  and  the  hundred's  columns  respectively.  The  answers 
to  the  problems  were  chosen  at  random  (excluding  zero) , and 
each  answer  was  arbitrarily  assigned  to  one  of  the  four  cate- 
gories of  carry-over.  Then  the  addends  were  chosen  so  that 
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the  problem  would  correspond  to  the  category  of  carry-over  to 
which  it  was  assigned. 

The  digits  of  the  addends  were  modified  to  provide  approx- 
imately equal  representation  of  each  digit  1-9  in  each  position 
of  the  sum,  with  the  following  exceptions:  1)  in  sums  with  two 
3-digit  addends,  the  number  ”9"  is  not  found  in  the  hundred's 
place  of  the  first  or  second  addends  and  large  numbers  are 
underrepresented  in  those  two  positions;  2)  in  sums  of  a 3- 
digit  addend  plus  a 2-digit  addend,  the  number  "9"  is  not  found 
in  the  hundred's  place  (of  the  first  addend)  and  large  numbers 
are  underrepresented  in  this  position. 

Variables . The  main  data  were  the  number  of  uncomputed 
(blank)  positions  in  the  reported  sum  and  the  number  of  correct 
positions  in  the  reported  sum.  Mean  performance  for  these 
measures  was  obtained  for  each  of  the  conditions. 

Sentence  Recall  Task.  The  sentence  recall  task  is  an 
original  task  designed  for  this  study.  It  embodies  much  of 
the  theoretical  essence  of  a free-recall  cluster  analysis  in 
studying  the  individual's  attempt  to  organize  incoming  informa- 
tion. In  the  task,  subjects  are  read  a series  of  simple  sentences, 
each  sentence  expressing  an  idea  or  attribute  relating  to  one 
of  four  topics.  At  the  end  of  the  series  subjects  are  to  con- 
tiguously recall  the  ideas  associated  with  each  topic.  In 
the  present  design,  the  number  of  ideas  associated  with  each 
topic  is  varied  (4,  5,  or  6). 

The  task  is  designed  to  get  subjects  to  integrate  the 
information  presented  in  the  simple  sentences  on  each  topic. 

To  the  extent  that  individuals  performed  such  integration,  they 
should  be  able  to  decrease  the  memory  load  requirements  of  the 
task.  The  major  operations  determining  performance  then  should 
be:  abstraction  (organization  and  integration  of  sentence 
information),  storage,  and  retrieval. 

Procedure.  In  this  task  subjects  had  to  recall  and  organize 
information.  Subjects  heard  a list  of  sentences  on  four 
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alternating  topics.  At  the  end  of  the  list  they  heard  a cue 
("RECALL")  and  had  approximately  two  minutes  to  recall  the 
information  in  the  sentences  and  organize  it  by  topic.  Subjects 
were  told  that  they  could  either  organize  the  information  into 
one  complex  sentence  or  use  phrases  to  describe  the  information 
they  had  heard. 

The  following  shows  the  sequence  of  events  for  part  of  a 
trial: 

STIMULI 

"The  party  was  loud." 

"The  snake  was  in  the  cage." 

"The  party  was  a surprise." 

"The  cage  was  glass." 

"The  party  was  a success." 

"The  snake  shook  its  rattles." 

"RECALL" 

RESPONSES 

The  loud  surprise  party  was  a success. 

OR 

Party  was  a surprise,  loud,  a success. 

The  snake  in  the  glass  cage  shook  its  rattles. 

OR 

Snake  in  cage  shook  rattles,  cage  was  glass. 

Stimuli  and  design.  Subjects  completed  one  practice  and 
three  critical  trials  of  this  task  each  day.  They  did  not 
receive  feedback  after  the  practice.  The  acquisition  list  for 
each  trial  was  based  on  four  topics.  Each  sentence  in  the 
acquisition  list  explained  a relationship  between  attributes 
of  one  of  the  four  topics.  Each  sentence  had  one  subject  and 
one  word  or  phrase  describing  the  subject  or  an  action  by  the 
subject.  The  sentences  were  presented  alternately  by  topic 
so  that  every  fourth  sentence  was  on  the  same  topic.  In  the 
first  trial  16  sentences  were  presented,  four  for  each  topic; 
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in  the  second  trial  20  sentences  were  presented,  five  for  each 
topic;  and  in  the  third  trial  24  sentences  were  presented,  six 
for  each  topic. 

Variables.  Two  principal  recall  measures  were  obtained 
for  the  task.  A cluster  recall  measure  consisted  of  the  number 
of  attributes  recalled  contiguously  for  each  topic.  This  mea- 
sure reflects  the  number  of  attributes  that  the  subject  correctly 
assigns  to  a given  topic.  The  item  recall  measures  consisted 
of  the  number  of  nouns,  adjectives,  and  verbs  (with  the  excep- 
tion of  TO  BE) that  were  recalled  from  a topic  description. 

Credit  was  given  in  both  measures  for  paraphrases  and  synonyms. 

Sentence  Recognition  Task.  The  sentence  recognition  task 
is  a replication  of  a task  designed  by  Bransford  and  Franks 
(1971)  to  investigate  the  acquisition  and  retention  of  linguis- 
tic ideas.  On  acquisition  in  their  study  subjects  were  presented 
sentences  containing  various  combinations  of  ideas  from  a com- 
plex, four-idea  sentence.  Retention  was  tested  with  a recogni- 
tion test  using  only  new  sentences  with  ideas  from  the  same 
complex  sentence.  Bransford  and  Franks  found  that  the  more 
ideas  in  a (new)  test  sentence  the  more  likely  the  subjects 
were  to  judge  that  sentence  as  old  with  high  confidence. 

The  present  task  added  a modification  to  the  Bransford 
and  Franks  design.  Where  Bransford  and  Franks  used  the  same 
syntax  in  all  the  derived  sentences  as  in  the  complex  sentence, 
we  have  varied  whether  the  syntax  is  the  same  or  different  in 
the  sentences.  This  modification  is  intended  to  show  the  extent 
to  which  the  effect  is  dependent  on  repetition  of  syntax. 

Performance  on  the  sentence  recognition  task  presumably 
depends  upon  the  individual  integrating  and  organizing  the 
ideas  from  the  various  sentences,  such  that  the  individual  can 
no  longer  distinguish  the  idea  combinations  that  were  presented 
from  those  that  were  the  product  of  integration  processes.  If 
syntas  does  not  completely  explain  the  effect,  then  whatever 
variance  remains  should  be  due  to  abstraction  operations  (idea 
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integration) , storage  operations  (retaining  sentences  in  memory) 
and  search  operations.  Search  operations  are  involved  as  opposed 
to  retrieval  operations  because  the  test  is  recognition. 

Procedure.  In  this  task,  subjects  decided  whether  or  not 
they  had  heard  particular  sentences  from  a list  of  sentences 
previously  presented.  An  acquisition  list  of  12  sentences 
based  on  two  topics  was  orally  presented.  Fourteen  recognition 
sentences  about  the  two  topics  were  then  presented.  After 
each  recognition  sentence  was  read,  subjects  decided  whether 
or  not  it  was  a member  of  the  acquisition  list  and  gave  a con- 
fidence rating  to  indicate  how  certain  he/she  was  that  his/her 
answer  was  correct.  Subjects  were  told  that  the  wording  of  a 
sentence  from  the  recognition  list  had  to  be  identical  to  the 
wording  of  the  corresponding  sentence  from  the  acquisition  list 
in  order  for  the  sentences  to  be  considered  the  same.  A prac- 
tice task  was  completed  before  each  administration  of  the 
critical  task,  but  no  feedback  was  provided. 

The  following  shows  the  sequence  of  events  of  part  of  a 

task. 

ACQUISITION  LIST 

"The  rock  rolled  down  the  mountain  crushing  the  hut  at  the  edge 
of  the  woods . " 

"The  actress  looked  at  her  reflection." 

"The  rock  that  rolled  down  the  mountain  crushed  the  hut." 

"The  actress  wearing  a hat  looked  in  the  mirror." 


RECOGNITION  LIST 

RESPONSE 

CONFIDENCE  RATING 

"The  actress  looked  at  her 
reflection. " 

(S) 

NO 

1 

"A  hut  was  at  the  edge  of  the  woods. 

" Yes 

© 

2 

"The  rock  that  rolled  down  the  moun- 
tain crushed  the  hut." 

(vy> 

No 

3 

"The  mirror  was  cracked." 

Yes 

© 

4 

f 


Stimuli  and  design.  Subjects  completed  this  task  twice 
each  day.  Each  task  was  based  on  two  primary  complex  sentences 
about  different  topics.  Each  primary  sentence  had  two  subjects. 
In  one  of  the  primary  sentences,  each  subject  had  two  modifying 
phrases;  in  the  other  primary  sentence  one  subject  had  three 
modifying  phrases  and  the  other  subject  had  one  modifying  phrase. 
For  example,  the  primary  sentence  "The  teenager  who  lives  next 
door  broke  the  large  window  on  the  porch"  has  two  subjects  and 
each  subject  has  two  modifying  phrases. 


The  teenager, 


\ 


lives  next  door. 

broke  the  window. 


The  window 


was  large. 


\ 


was  on  the  porch. 


In  each  of  the  remaining  sentences  in  the  task,  one  or  more  of 
the  modifying  phrases  which  were  found  in  the  primary  sentence 
was  deleted.  For  example,  some  sentences  generated  from  the 
above  primary  sentence  are; 

The  teenager  broke  the  window. 

The  large  window  was  on  the  porch. 

The  teenager  who  lives  next  door  broke  the  large  window. 


In  each  task,  the  sentences  generated  from  one  of  the  primary 
sentences  used  the  same  wording  as  the  primary  sentence.  This 
was  called  a consistent  set  of  sentences.  The  example  above 
is  a set  of  consistent  sentences.  The  sentences  generated 
from  the  other  primary  sentence  in  the  task  were  inconsistent. 
For  example,  the  following  is  a primary  sentence  with  a set  of 
inconsistent  generatives: 

PRIMARY  SENTENCE  GENERATIVES 

The  executive  who  drank  the  hot  The  executive  was  sitting  at 
coffee  at  his  desk  had  his  his  desk, 

briefcase. 

At  his  desk,  the  executive  was 
drinking  hot  coffee. 


The  coffee  was  hot  and  was  drunk 
by  the  executive  with  the 
briefcase . 
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Of  the  14  sentences  in  the  recognition  list,  two  were  the  two 
primary  sentences,  six  were  sentences  from  the  acquisition  list 
("OLD")  and  six  were  sentences  the  subjects  had  not  heard  before 
("NEW").  Thus,  both  the  acquisition  list  and  the  recognition 
list  contained  twelve  non-primary  sentences,  six  on  each  of 
two  topics;  of  the  six,  half  were  "OLD"  and  half  were  "NEW". 

Of  the  six  generatives  of  a particular  topic,  two  were  made  by 
deleting  one  modifying  phrase  from  the  primary  sentence,  two 
were  made  by  deleting  two  modifying  phrases  from  the  primary 
sentence,  and  two  were  made  by  deleting  three  modifying  phrases 
from  the  primary  sentence.  In  the  acquisition  list,  the  topics 
of  the  twelve  sentences  were  alternated,  that  is,  every  other 
sentence  was  a generative  of  the  same  primary  sentence  and  no 
generative  was  based  on  the  same  primary  sentence  as  the 
previous  generative.  In  the  recognition  list,  the  generatives 
plus  the  two  primary  sentences  were  presented  in  random  order. 

Variables.  In  this  task  the  data  were  the  yes-no  judgments 
and  the  ratings  of  those  judgments.  The  information  was  com- 
bined in  a modified  rating  scale  by  giving  all  ratings  for  the 
subject's  "old"  responses  a "plus"  and  all  ratings  for  "new" 
responses  a "minus".  Thus,  a 10-point  rating  scale  was  developed 
ranging  from  minus  five  to  plus  five  (excluding  zero) . Using 
this  scale,  the  best-fitting  linear  functions  relating  rating 
and  sentence  complexity  were  derived  for  old  and  new  sentences. 
Mean  number  of  errors  and  the  overall  frequency  distribution  of 
the  ratings  were  also  obtained. 

Letter  Rotation  Task.  This  task  replicates  a task  inves- 
tigated extensively  by  Lynn  Cooper  and  Roger  Shepard  (197  3)  . 

The  subject  is  shown  two  items  and  is  asked  to  determine  whether 
they  are  different  objects  or  the  same  object  from  a different 
perspective.  It  has  been  found  that  the  time  required  to  answer 
this  question  is  a linear  function  of  the  degree  of  rotation 
required  to  bring  the  figures  to  a common  orientation. 
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With  respect  to  the  operations  discussed  earlier,  this 
task  would  involve  the  transformation  and  comparison  operations. 
The  rotation  of  the  objects  would  involve  processes  subsumed 
under  transformations.  Once  the  objects  have  been  rotated  the 
comparison  operation  is  required  to  determine  if  the  objects 
are  the  same  or  different. 


Procedure.  In  this  task,  subjects  decided  if  two  letters 
were  the  same  or  different  using  two  rules.  According  to  Rule  1, 
two  letters  were  the  same  if  a subject  could  slide  one  letter 
on  top  of  the  other  and  they  matched  exactly.  For  example, 
using  Rule  1 LL  is  "SAME"  and  JL  is  "DIFFERENT".  Subjects 
completed  the  first  page  of  the  task  using  only  Rule  1.  Subjects 
were  instructed  to  use  Rule  2 for  pages  2,  3,  and  4.  According 
to  Rule  2,  two  letters  were  the  same  if,  after  mentally  rotating 
one  letter,  a subject  could  slide  one  letter  on  top  of  the  other 
and  they  matched  exactly.  For  example,  using  Rule  2 
and  “■)  are  "SAME"  and  \s  and  LF  are 

"DIFFERENT".  Subjects  were  given  thirty  seconds  to  work  on  the 
first  page  of  this  task  and  sixty  seconds  to  work  on  the  second, 
third,  and  fourth  pages  of  this  task. 


The  following  are  examples  of  trials  used  in  this  task: 


D Q 
1 * 


* Jl 


B 


Same  Diff. 

Same  Diff. 

B_ 

_ a _ 

/>•  «<\  _ 

_o  0_ 

</  _ 

_>>  b _ 

\>-  F _ 

_ G © _ 

_ R 

b _ 

Same  Diff. 

■</\y 

_C0_ 

_a 

_e  © _ 
_ 
o _ 
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Stimuli  and  design.  This  task  was  administered  once  each 
day.  During  each  administration  subjects  completed  four  pages. 
Each  page  had  four  columns  of  18  trials  each.  The  stimuli  were 
pairs  of  capital  F's,  G's,  and  R's.  In  half  of  the  pairs  of 
letters,  the  second  letter  was  a mirror  image  of  the  first, 
making  it  impossible  to  slide  the  letters  on  top  of  one  another 
and  have  them  match.  These  pairs  were  "DIFFERENT".  In  the 
rest  of  the  pairs  of  letters  one  letter  was  not  a mirror  image 
of  the  other,  so  they  were  "SAME".  In  the  second,  third,  and 
fourth  pages  of  the  task  the  second  letter  was  rotated  with 
respect  to  the  first  letter.  On  the  second  page,  the  second 
letter  of  each  pair  had  a 60°  rotation  with  respect  to  the 

O 

first  letter,  on  the  third  page  the  second  letter  had  a 120 
rotation  with  respect  to  the  first  letter.  And  on  the  fourth 

O 

page  the  second  letter  had  a 180  rotation  with  respect  to  the 
first  letter.  For  example,  on  the  third  page,  if  the  first 
letter  was  oriented  at  an  angle  of  60°  clockwise  with  respect 
to  the  vertical  axis,  the  second  letter  would  be  oriented  at 

O 

180  with  respect  to  the  vertical  axis.  The  left  member  of 
a pair  was  in  one  of  six  positions  clockwise  with  respect  to 
vertical:  0°  (vertical),  60°,  120°,  180°,  240°,  or  300°.  Since 
three  letters  were  used,  each  item  was  one  of  72  possible  com- 
binations for  a particular  page.  The  items  were  distributed 
randomly  within  a page,  with  the  restriction  that  in  each 
column  50%  of  the  items  were  "SAME"  and  50%  were  "DIFFERENT". 

Variables.  The  data  consisted  of  the  mean  time  per  item 
for  the  same  - different  judgments  across  the  varying  degrees 
of  rotation.  The  percentage  of  errors  in  each  condition  was 
also  computed. 

Physical  Match  Task.  The  physical  match  task  is  a "same- 
different"  classification  task  derived  from  an  experimental 
paradigm  developed  by  Posner  and  Mitchell  (1967).  The  subject 
is  presented  two  uppercase  letters  and  asked  to  respond  "same" 
if  the  two  letters  are  physically  identical  and  "different" 
otherwise.  In  terms  of  our  operations,  this  task  represents 
an  uncomplicated  assessment  of  the  comparison  process. 
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Procedure.  Subjects  saw  two  letters  side  by  side  and 
classified  the  pair  as  "SAME"  or  "DIFFERENT".  A pair  of  letters 
was  to  be  classified  as  "SAME"  if  the  items  in  the  pair  were 
the  same  letter  and  "DIFFERENT"  if  the  items  were  not  the  same 
letter.  Subjects  completed  a non-timed  practice  task  and  then 
were  given  45  seconds  to  work  on  the  critical  task. 

The  following  shows  the  events  of  7 trials: 

RESPONSES 

Same  Diff. 

/ E E 

/ T T 

E H / 

/ H H 

T E / 

/ A A 

E A / 


STIMULI 

Same  Diff. 

E E ' 

T T 

E H 

H H 

T E 

A A 

E A 


0 

I. 

i: 

0 

t 

1 

i 


Stimuli  and  design.  This  task  occurred  once  during  each 
session  (day) . The  stimuli  were  combinations  of  the  letters 
A,  E,  H,  and  T.  The  task  was  constructed  so  that  half  of  the 
stimuli  had  a correct  response  of  "SAME"  and  half  had  a correct 
response  of  "DIFFERENT".  The  various  letter  combinations 
occurred  randomly  with  the  restriction  that  a particular  answer 
not  occur  more  than  four  times  consecutively. 

Variables.  The  data  consisted  of  the  mean  time  per 
for  the  same  - different  judgments.  The  percentage  of  errors 
was  also  obtained. 

Set  Membership  Task.  The  set  membership  task  replicates 
an  experimental  paradigm  developed  by  Sternberg  (1967,  1969) 
to  study  information  retention.  Subjects  memorized  a sub-span 
list  of  items  (1-4  letters)  and  had  to  respond  to  a series 
of  probes  as  to  whether  the  probe  was  in  the  memory  set.  Sub- 
jects made  "yes"  or  "no"  responses  to  each  item.  Time  to 
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process  and  respond  to  the  probe  increased  linearly  as  the  size 
of  the  memory  set  increased. 

This  task  involves  two  types  of  operations: 

(1)  Storage  operations  are  required  to  maintain  the 
memory  set  in  memory. 

(2)  Search  operations  are  required  to  scan  the  target 
set  for  the  probe. 

Procedure . In  this  task,  subjects  decided  if  letters 
presented  in  printed  columns  were  members  of  a target  set. 
Subjects  were  instructed  to  put  a check  to  the  left  of  a stim- 
ulus letter  (in  a column  labeled  "YES")  if  the  letter  was  a 
member  of  a target  set  and  to  put  a check  to  the  right  of  a 
stimulus  letter  (in  a column  labeled  "NO")  if  the  letter  was 
not  a member  of  the  target  set.  Subjects  were  verbally  pre- 
sented with  a target  set  of  1,  2,  3,  or  4 letters  and  were 
then  given  thirty  seconds  to  work  on  the  task.  After  thirty 
seconds  subjects  were  told  to  stop  working  and  to  write  the 
target  letter (s)  at  the  bottom  of  the  page. 

The  following  shows  the  sequence  of  events  for  eight  items 
of  a task: 


TARGET 


STIMULUS 


RESPONSE 


M /_ 
/ N 


Stimuli  and  design.  Subjects  completed  two  practice  and  12 
critical  trials  of  this  task  each  day.  They  received  no  feed- 
back on  their  performance.  Both  the  stimuli  and  the  targets 
were  randomly  chosen  from  the  set  of  19  letters  exclusive  of 
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vowels,  Y and  W.  Target  sets  of  one,  two,  three,  and  four 
letters  occurred  randomly  with  the  restriction  that  no  two 
consecutive  targets  had  the  same  number  of  letters.  Half  of 
the  letters  in  each  stimulus  list  were  members  of  the  target 
set  for  that  list,  and  the  order  in  which  the  letters  occurred 
was  random. 

Variables.  The  data  consisted  of  the  mean  time  to  process 
each  probe.  These  times  were  plotted  against  the  number  of 
elements  in  the  target  set  and  the  best-fitting  linear  function 
derived.  The  percentage  of  errors  was  also  computed. 

Scan  and  Search  Task.  The  scan  and  search  task  was  origi- 
nally developed  by  Ulric  Neisser  (1967)  to  assess  the  memory 
search  process.  Subjects  search  a list  of  letters  for  target 
letters.  The  list  contains  a column  of  letters  with  four  to 
six  letters  to  a row.  Here  the  subject  searches  down  the 
column  and  responds  only  when  a target  letter  is  found,  as 
opposed  to  the  set  membership  task  where  the  subject  responds 
to  every  item.  Neisser  found  that  search  time  increased  the 
further  down  the  coJumn  the  target  letter  was  positioned.  Also, 
and  most  important  to  us,  he  found  that  with  extensive  practice, 
his  subjects  could  search  for  any  one  cf  ten  targets  simultaneously 
with  the  same  speed  as  they  could  search  for  one  target.  In 
other  words,  the  size  of  the  target  set  did  not  affect  search  times. 


1 

l 

I 

I 

I 


In  the  present  version  of  the  task,  subjects  memorize  a 
sub-span  target  set  (1,  2,  or  4 letters)  and  then  search  a list 
of  letters  for  target  letters.  Performance  is  tested  on  "clear" 
and  "degraded"  lists.  Individual  performance  should  depend 
upon  storage  operations  to  hold  the  target  set  in  memory  and 
search  operations  to  scan  the  column  for  the  target.  A com- 
parison between  the  "clear"  and  "degraded"  conditions  should 
reveal  effects  of  the  encoding  operation  since  these  processes 
would  be  required  to  "clean  up"  the  degraded  stimuli. 

Procedure . In  this  timed  task,  subjects  searched  columns 
of  groups  of  letters  looking  for  particular  letters.  At  the 
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| beginning  of  each  trial,  a target  set  of  one,  two,  or  four 

letters  was  verbally  presented.  Subjects  then  searched  for 
I items  which  contained  one  of  the  target  letters,  placing  a 

check  next  to  those  that  did.  Subjects  received  20  seconds 
j for  the  one-letter  target  condition  and  30  seconds  for  the  two 

and  four-letter  target  conditions.  At  the  end  of  each  trial, 
each  subject  drew  a line  under  the  last  item  he  had  looked  at 
and  wrote  down  the  letters  he  had  been  looking  for. 
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The  following 

TARGET 
B,  F 


re  events  of  part  of 

STIMULUS 
A U M F Z 
L N D Q G 
K L J D P 
Y K T M F 
X V D B Y 


a typical  trial: 

RESPONSE 
A U M F Z / 

L N D Q G 

K L J D P 

Y K T M F / 

X V D B Y / 


Stimuli  and  design.  This  task  was  completed  twice  each 
day.  Subjects  received  two  practice  and  six  critical  trials 
during  each  administration.  No  feedback  was  provided.  Half 
of  the  stimulus  pages  were  degraded  by  printing  them  through  a 
series  of  filters.  The  filters  were  Xerox  No.  8R535  Document 
Carrier  and  Screens,  and  were  made  of  a regular  pattern  of 
small  white  dots.  During  an  administration,  subjects  completed 
the  non-degraded  one-letter,  two-letter,  and  four-letter  tar- 
get conditions,  in  that  order,  and  then  completed  the  degraded 
targets  in  the  same  order. 


Each  item  was  a string  of  five  capital  letters.  There 
*•  were  four  columns  of  16  items  each  on  each  page.  Fifty  percent 

^ of  the  items  in  each  column  contained  a target  letter  and  no 

L item  contained  more  than  one  target  letter. 
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Variables . The  principal  measure  was  the  mean  time  to 
scan  each  item  and  make  the  yes-no  judgments.  These  times  were 
plotted  against  the  number  of  items  in  the  search  set  and  the 
best-fitting  linear  function  derived.  Separate  functions  were 
derived  for  the  clear  and  degraded  conditions.  The  percentage 
of  errors  for  each  condition  was  also  computed. 
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METHOD 


Subjects 

Subjects  were  68  female  and  male  students  who  answered 
an  advertisement  in  the  school  newspaper  at  Georgetown  Univer- 
sity. They  were  paid  for  their  participation  in  the  study  and 
were  tested  in  groups  ranging  from  three  to  eight  people. 

General  Procedure 

The  subjects  participated  in  two  testing  sessions,  each 
approximately  three  hours  in  duration  and  scheduled  a day  apart 
All  eight  tasks  were  presented  in  the  same  order  each  session. 
Where  appropriate,  different  stimuli  were  used  in  each  testing 
session  and  the  order  of  stimuli  in  the  task  was  randomized. 

At  the  beginning  of  the  testing  sessions  subjects  received 
a booklet  containing  the  instructions  and  response  sheets  for 
all  the  tasks.  Appendix  A presents  instructions  for  each  task 
and  selected  stimuli  and  response  sheets.  The  experimenter 
paced  subjects  through  this  booklet  as  the  subjects  performed 
each  task  in  turn.  For  each  task  the  following  events  occurred 
Instructions  for  the  task  were  played  from  cassette  tapes  as 
the  subjects  read  along.  Following  the  instructions,  the 
experimenter  answered  questions  and  administered  a practice 
sample  of  the  task.  This  practice  sample  served  the  function 
of  introducing  the  subject  to  the  task  and  providing  some  "warm 
up"  prior  to  the  critical  task.  Performance  on  the  practice 
task  was  monitored  to  ensure  that  subjects  understood  what  the 
task  would  require.  The  critical  task  was  administered  upon 
completion  of  the  practice  sample. 
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Data  Analysis 

The  general  analytical  plan  consisted  of  two  stages.  The 
first  stage  was  primarily  concerned  with  analyses  of  the 
individual  tasks.  Each  task  was  examined  to  determine  whether 
the  expected  (from  previous  findings)  or  hypothesized  (based 
on  "new"  treatments)  phenomena  actually  occurred.  This  first 
stage  was,  in  essence,  a "forms  check"  for  our  particular 
implementations . 

As  a general  analysis,  analysis  of  variance  (ANOVA)  was 
used  in  this  stage.  The  purpose  of  the  ANOVA  was  to  describe 
and  confirm  the  previous  findings  on  each  task,  namely,  the 
pattern  of  significant  and  nonsignificant  effects  of  the  treat- 
ments on  overall  task  performance.  In  addition,  since  some  of 
the  tasks  included  repetitions  within  a day,  it  was  possible 
to  test  treatment-by-subject  interactions.  Significant  treat- 
meny-by-subject  interactions  would  mean  that  subjects  responded 
differently  to  the  treatments;  therefore,  this  interaction 
effect  would  indicate  that  further  study  would  be  required  in 
order  to  identify  two  or  more  parameters  for  use  in  describing 
subjects’  differences.  : 

An  ANOVA  for  each  task  was  performed  both  on  the  raw  data 
and  where  appropriate,  on  the  transformed  scores.  The  reason 
for  the  data  transformation  was  that  some  of  the  tasks  had  a 
limited  range  of  possible  reaponse  scores.  On  those  tasks, 
means  and  variances  would  be  correlated,  which  would  violate  the 
assumption  of  ANOVA.  Moreover,  the  limited  ranges  were  likely 
to  produce  interaction  effects  among  treatments.  For  those 
tasks,  arcsin  or  log  transformations  were  used  (Winer,  1962) . 

The  reason  for  analyzing  the  raw  data  was  to  increase  under- 
standing of  parameters  derived  from  the  raw  data.  Previous 
research  has  derived  parameters  from  the  raw  data  and  not  from 
the  transformed  scores.  Therefore,  it  was  necessary  to  perform 
ANOVAs  on  the  raw  data  in  order  to  compare  the  results  to  those 
from  previous  studies. 
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The  second  stage  of  the  analysis  was  to  estimate  individuals' 
parameters  on  each  task,  such  as  slopes  and  intercepts,  based 
upon  the  results  of  the  ANOVAs.  The  selection  of  parameters 
to  be  estimated  (e.g.,  slopes  and  intercepts)  was  dependent 
upon  significant  effects  from  the  ANOVAs.  After  estimating 
parameters,  they  were  correlated  with  each  other.  In  theory, 
the  pattern  of  correlations  would  show  higher  correlation 
coefficients  among  those  parameters  which  involve  the  same 
information  processing  operations,  thereby  providing  evidence 
for  the  construct  validity  of  the  operation. 
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RESULTS  AND  DISCUSSION 


Group  Measures 

Overview.  The  results  of  several  different  types  of  analyses 
are  presented  below.  The  first  set  deals  with  the  general 
results,  or  group  effects,  for  each  of  the  eight  tasks  individ- 
ually. In  general,  these  results  are  evaluated  in  terms  of  the 
degree  to  which  they  replicate  previous  findings  using  similar 
paradigms.  For  those  tasks  to  which  "new"  treatments  or  condi- 
tions have  been  added,  more  detailed  analyses  have  been  conducted. 
These  analyses  describe  and  elaborate  upon  the  hypothesized 
phenomena  demonstrated  by  the  results.  More  specifically,  three 
of  the  tasks,  namely  Physical  Match,  Set  Membership,  and  Letter 
Rotation,  are  direct  adaptations  of  tasks  previously  investigated 
as  part  of  this  research  program  and  have  been  demonstrated  to 
be  potentially  valuable  as  candidate  tasks  for  the  test  battery. 
Their  use  in  the  current  study  will  be  essentially  to  investi- 
gate the  replicability  of  previous  findings  using  different 
materials  and  formats.  Two  other  tasks,  namely  Scan  and  Search 
and  Mental  Addition,  are  direct  adaptations  of  paradigms  used 
by  other  investigators  who  were  not  primarily  concerned  with 
issues  relating  to  test  construction  or  individual  differences. 
Thus,  for  these  tasks,  a principal  concern  is  again  the  demon- 
stration of  replicability  of  the  major  phenomena.  In  addition, 
all  five  of  these  tasks  are  evaluated  in  later  sections  as  to 
their  value  for  the  development  of  individual  difference  para- 
meters. 

Of  the  remaining  three  tasks,  two  of  them,  namely  Letter 
Recall  and  Sentence  Recognition,  while  derived  from  paradigms 
in  the  literature,  are  sufficiently  unique  in  this  implementa- 
tion to  merit  fairly  detailed  examination.  Analyses  and 
arguments  are  presented  which  describe  the  principal  treatment 
effects  and  which  discuss  the  hypothesized  nature  of  the  under- 
lying operations.  The  final  task,  Sentence  Recall,  is  of  a 
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slightly  different  sort:  it  has  been  developed  primarily  as  a 
potential  source  of  individual  difference  parameters.  The 
"group  effects"  per  se,  although  presented,  are  not  particularly 
germane  to  the  evaluation  of  the  usefulness  of  this  task  with 
respect  to  individual  measures. 

Physical  Match 

This  task  is  a partial  replication  of  the  Posner  and 
Mitchell  (1967)  paradigm  which  was  also  included  in  the  previous 
study  (Rose  and  Fernandes,  1977)  . The  main  purpose  for  its 
inclusion  in  the  present  study  was  to  ascertain  whether  a paper- 
and-pencil  format  would  produce  baseline  (i.e.,  physical  match) 
performance  compatible  with  previous  findings  from  studies  which 
used  alternate  testing  procedures.  Such  a result  would  poten- 
tially increase  the  flexibility  of  administration  for  a test 
battery. 

Posner  and  Mitchell  (op.  cit.)  reported  a mean  "same" 
physical  match  response  time  of  549  msec.  The  equivalent 
condition  in  the  Rose  and  Fernandes  (op.  cit.)  study  resulted 
in  a mean  response  time  of  585  msec  for  Day  1 and  547  msec  for 
Day  2.  The  mean  processing  time  per  item  in  the  present  study 
was  713  msec  on  Day  1 and  579  msec  on  Day  2.  In  terms  of  other 
descriptive  measures,  the  present  implementation  produced 
slightly  larger  standard  deviations  and  ranges  than  the  Rose 
and  Fernandes  study  [i.e.,  s.d.  = 80  msec  (present),  57  msec 
(Rose  and  Fernandes);  range  = 480  msec  (present),  277  msec 
(Rose  and  Fernandes)].  Overall  error  rate  was  less  than  1%. 
Thus,  although  no  formal  statistical  methods  were  used  for 
evaluating  the  compatibility  of  findings,  it  is  apparent  that 
the  results  obtained  by  the  second  day  of  testing  are  of  the 
same  order  of  magnitude  as  those  previously  reported. 

Set  Membership 

This  task  is  a replication  of  the  Sternberg  (1975)  paradigm 
which  was  also  included  in  the  Rose  and  Fernandes  (1977)  study. 
As  was  the  case  for  the  Physical  Match  task,  the  primary  con- 
sideration was  to  determine  if  the  present  implementation  would 
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result  in  effects  compatible  with  other  testing  formats.  The 
basic  finding  to  be  replicated  is  the  demonstration  of  a linearly- 
increasing  function  relating  time  per  item  to  set  size.  Figure  1 
shows  this  relationship  for  the  present  study,  for  each  day  of 
testing.  As  can  be  seen,  the  expected  result  was  obtained. 

This  indicates  that  the  processing  of  the  memory  set  was  serial 
and  exhaustive,  as  was  true  in  the  previous  studies. 

In  terms  of  the  parameters  of  the  function  (slope  and  inter- 
cept) , the  results  from  the  three  studies  are  highly  similar. 

For  example,  on  Day  1 in  the  Rose  and  Fernandes  study,  the 
obtained  slope  and  intercept  were  63  msec  and  496  msec,  respec- 
tively; in  the  present  study,  these  values  were  60  msec  and 
500  msec.  Sternberg  (1975) , using  more  practiced  subjects, 
obtained  values  of  about  38  msec  (slope)  and  about  400  msec 
(intercept) . For  other  descriptive  measures,  the  present  study 
and  the  Rose  and  Fernandes  results  are  very  similar.  For  example, 
for  comparable  slope  measures,  the  obtained  standard  deviations 
and  ranges  are  21  msec  and  97  msec  (Rose  and  Fernandes)  versus 
20  msec  and  110  msec  (present  study,  Day  2) . For  the  inter- 
cept measure,  the  standard  deviations  and  ranges  are  78  msec 
and  471  msec  (Rose  and  Fernandes)  versus  69  msec  and  374  msec 
(present  study.  Day  2) . Proportions  of  errors  were  less  than 
2%  for  both  studies.  Thus,  we  are  confident  that  a paper-and- 
pencil  version  of  this  task  produces  results  which  are  equiva- 
lent to  previous  implementations. 

Letter  Rotation 

This  particular  adaptation  of  a paradigm  developed  by 
Shepard  and  co-workers  (e.g. , Shepard  and  Metzler,  1971;  Snyder, 
1972)  was  previously  employed  by  Rose  (1974),  using  the  identical 
stimuli  and  response  formats.  The  primary  empirical  finding  of 
interest  is  the  previously  observed,  monotonically-increasing 
function  relating  time  per  item  and  degrees  of  stimulus  rotation 
required.  There  has  been  some  controversy  in  the  literature 
as  to  whether  this  function  is  linear  or  bow-shaped  and  whether 
the  function  reflects  the  subjects  truly  "mentally  rotating" 
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V ( Day  1 ) = .063X  + .496 


069X  + .374 


Target  Set  Size  (letters) 


Set  Membership:  Processing  Time  per  Item  for  the  Letters 
in  the  Target  Set. 


a visual  representation  or  performing  some  other  operation. 
However,  there  is  no  disagreement  (and  numerous  replications) 
regarding  the  observed  monotonic  increases.  With  regard  to 
the  "true"  nature  of  the  operations  involved,  this  particular 
implementation  avoids  the  controversy  by  instructing  the  sub- 
jects to  mentally  rotate  the  letters.  This  instruction  pre- 
sumably limits  the  use  of  alternate  strategies. 

Figure  2 shows  the  mean  processing  time  per  item  for  the 
four  degrees  of  rotation.  As  can  be  seen,  the  basic  result 
was  obtained:  the  time  per  item  monotonically  increases  with 
the  degrees  of  required  rotation.  The  shapes  of  these  functions 
are  clearly  bow-shaped,  with  the  larger  degrees  of  rotation 
showing  faster  than  expected  response  times.  Without  getting 
into  the  controversies  concerning  the  bow-shaped  finding,  the 
simplest  explanation  is  a within-day  practice  effect:  subjects 
performed  these  conditions  in  increasing  order  of  difficulty. 
Thus,  the  latter  conditions  would  have  benefited  from  earlier 
practice.  Also,  the  fact  that  all  the  trials  for  a given 
rotation  were  presented  at  the  same  time  probably  enabled 
subjects  to  improve  performance  if  compared  to  a randomly 
presented  stimulus  set.  If  it  could  be  assumed  that  more 
difficult  conditions  would  benefit  more  than  easier  conditions, 
a bow-shaped  function  could  be  predicted. 

These  results  (including  the  bow-shaped  function)  mimic 
in  all  important  aspects  the  findings  of  Rose  (1974) . In  both 
experiments,  the  obtained  slopes  (approximately  15  msec  per 
degree) , standard  deviations  (less  than  10  msec  per  degree) 
and  ranges  (approximately  40  msec  per  degree)  were  virtually 
identical.  Also,  no  subject  in  either  study  produced  a nega- 
tive slope.  Thus,  while  these  results  suggest  a future  methodo- 
logical adjustment  (i.e.,  the  use  of  randomly-presented  stimuli) 
the  primary  empirical  result  was  obtained;  furthermore,  the 
paradigm  has  retained  its  value  as  a potential  source  of  impor- 
tant individual-difference  parameters. 
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Degrees  of  Rotation 


Figure  2.  Letter  Rotation:  Processing  Time  per  Item  for  the  Varying 
Degrees  of  Rotation. 


Scan  and  Search 


This  task  was  a variation  of  Neisser's  (1967)  procedure 
for  estimating  scanning  rate.  In  the  present  implementation, 
the  task  is  a direct  replication  of  one  used  by  Rose  (1974) , 
with  the  addition  of  a degraded  stimulus  condition.  Thus, 
there  were  two  major  reasons  for  including  this  task:  first, 
to  confirm  previous  findings  concerning  the  effect  of  set  size 
on  search  rate;  and  second,  to  explore  the  effects  of  an  addi- 
tional condition  (clear  and  degraded  stimuli) , both  as  an 
indicant  of  additional  processing  demands  and  as  a source  of 
individual  difference  parameters  that  would  reflect  these 
additional  demands. 

It  should  be  noted  that  the  original  Neisser  paradigm  was 
different  in  focus  from  the  current  task,  primarily  in  terms 
of  the  "role"  that  target  set  size  plays  in  the  theoretical 
analysis  of  performance.  For  example,  one  of  his  most  widely- 
cited  findings  is  that,  for  well-practiced  subjects,  target  set 
size  was  not  of  direct  importance.  Conversely,  the  current 
task  is  directly  concerned  with  this  variable;  potential  individ- 
ual difference  parameters  reflecting  searcn  operations  in  part 
depend  upon  finding  increasing  times-per-item  as  a function  of 
target  set  size.  Furthermore,  since  provision  for  extensive 
practice  is  not  feasible  within  the  context  of  a test  battery, 
it  is  preferable  to  consider  performance  as  occurring  at  the 
subject's  initial  exposure  to  the  task.  It  might  be  possible 
to  develop  a "degree  of  parallel  processing"  measure  for  each 
subject  (e.g.,  a measure  of  the  decrease  in  slope  across 
repeated  administrations) , but  again  we  prefer  to  evaluate 
this  task  as  a "static"  assessment. 

The  mean  processing  time  per  item  for  the  three  levels  of 
target  set  size  is  shown  in  Figure  3.  There  is  a very  strong 
effect  showing  processing  time  per  item  to  increase  with  target 
set  size.  The  figure  also  reveals  that  degraded  stimuli  tend 
to  increase  the  processing  time  per  item.  There  is  an  inter- 
action between  the  target  set  size  and  the  type  of  stimuli 
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Figure  3.  Scan  and  Search:  Processing  Time  per  Item  for  the  Letters  in  the 
Target  Set. 


(see  Appendix  B)  suggesting  that  processing  time  on  the  four- 
letter  target  set  was  least  affected  by  the  level  of  stimulus 
degradation.  Also,  the  data  demonstrated  that  practice  tends 
to  be  differentially  effective,  decreasing  processing  time  most 
for  the  four-letter  target  set.  These  data  suggest  that  at  the 
stage  of  practice  used  in  this  study,  subjects  are  serially 
processing  items  in  the  target  set,  and  that  as  practice 
increases  subjects  approach  a parallel  processing  mode. 

As  might  be  expected,  results  from  the  clear  stimulus 
conditions  are  virtually  identical  to  those  obtained  by  Rose 
(1974).  For  example,  the  slope  measure  (excluding  errors)  in 
the  present  study  has  a mean  of  330  msec  per  item,  a standard 
deviation  of  90  msec  and  a range  of  38  msec  per  item.  Com- 
parable results  from  Rose  (op.  cit..  Day  2)  are  338  msec  per 
item  (mean) , 86  msec  (standard  deviation) , and  48  msec  per 
item  (range) . 

Mental  Addition 

This  task  was  an  adaptation  of  Hitch's  (1978)  paradigm; 
as  such,  a primary  concern  of  the  present  study  was  to  repli- 
cate the  previously  reported  results.  Of  principal  interest 
in  this  regard  would  be  the  demonstration  of  an  increasing 
number  of  errors  with  increasing  number  of  carry  operations. 

In  addition,  this  task  was  included  to  test  an  extension  of 
the  theory  concerning  processing  requirements.  Further  evidence 
for  the  presence  of  storage,  retrieval,  and  transformation  opera- 
tions could  be  obtained  by  a slightly  different  casting  of  the 
observed  data.  For  example,  a "3  + 2,  carry  in  tens  and  hundreds” 
problem  hpothetically  requires  seven  items  be  stored  at  some 
time:  the  five  digits  and  the  two  carry  "markers".  Likewise, 

a " 3+3,  carry  in  tens"  problem  requires  storage  of  seven 
items:  the  six  digits  and  the  carry  "marker".  Using  similar 

logic,  all  the  problems  can  be  categorized  as  requiring  memory 
storage  for  5,  6,  7,  or  8 items.  If  the  hypothesized  operations 
are  important  to  task  performance,  then  performance  should 
decrease  (due  to  the  constant  5-second  deadline)  as  processing 
requirements  increase. 
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Evidence  for  the  replication  issue  is  shown  in  Figures 
4 and  5.  These  figures  differ  with  respect  to  the  dependent 
variable  plotted;  since  the  present  implementation  employed  a 
deadline  condition  rather  than  a response  time  measure  (as 
Hitch  used) , it  was  not  apparent  whether  a mean  number  of 
correct  positions  or  the  number  of  blank  positions  was  more 
appropriate  as  a comparison  measure  to  Hitch's  results.  In 
fact,  both  measures  reflect  Hitch's  findings  in  every  important 
aspect:  the  problems  with  larger  addends  produced  more  errors 

(fewer  correct  positions;  more  blanks)  and  errors  increase  with 
an  increase  in  number  of  carry  operations.  Of  particular  interest 
is  the  indication  of  a "flattening"  of  these  functions  at  the 
"tens"  and  "hundreds"  points.  This  was  also  reported  in  the 
Hitch  study.  Thus,  for  purposes  of  replication,  there  is  good 
reason  to  believe  that  subjects  in  the  present  study  performed 
the  task  in  much  the  same  manner  as  Hitch's  subjects. 

Regarding  our  extended  interpretation  of  the  results, 

Figures  6 and  7 present  the  two  dependent  variables  as  a func- 
tion of  the  hypothesized  number  of  operations  (storage  require- 
ments) required.  In  this  interpretation,  problems  that  were 
previously  considered  separately  were  combined  in  the  manner 
discussed  above;  for  example,  a "3  + 2,  carry  in  tens",  "3  + 2, 
carry  in  hundreds",  and  a 3 + 3,  no  carry"  were  considered  as 
equivalent.  As  can  be  seen  in  Figures  6 and  7,  the  predicted 
behavior  is  indeed  obtained.  A Scheffe  test  found  that  problems 
requiring  storage  for  six  items  did  not  differ  significantly 
among  themselves,  nor  did  problems  requiring  storage  of  seven 
items.*  Another  rather  startling  finding  is  the  obvious  linear- 
ity of  these  functions  (especially  for  "number  of  blanks"  per- 
formance) . This  linearity  would  perhaps  argue  for  a strict, 
linear-additive  model  of  the  operations  involved,  or  alterna- 
tively that  the  operations  necessary  for  a "carry"  step  are 
equivalent  in  processing  requirements  to  those  necessitated 
by  an  additional  digit  in  the  problem.  Although  fits  to  models 
developed  by  Hitch  (op.  cit.)  were  not  attempted,  these 

*For  blanks,  critical  difference  = .204,  a = .01,  df  = 7,  201; 
for  correct  positions,  critical  difference  = .253,  a = .01 
df  = 7,201. 
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Figure  4.  Mental  Addition:  Mean  Number  of  Correct  Positions  in  the 
Reported  Sums  for  the  Four  Types  of  Carry  Operations. 
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Figure  6.  Mental  Addition:  Number  of  Correct  Positions  in  Reported  Sums 
for  the  Derived  Levels  of  Memory  Load. 
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Figure  7.  Mental  Addition:  Number  of  Blank  Positions  for  the  Derived  Levels 
of  Memory  Load 
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analyses  and  results  provide  very  strong  evidence  that  our 
recasting  of  the  data  could  potentially  provide  "good"  measures 
of  individual  differences  in  information-processing  operations. 
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It  is  worth  mentioning  that  while  both  behavioral  measures, 
number  of  correct  positions  and  number  of  blank  positions, 
reflect  the  underlying  operations,  there  is  reason  to  think 
that  the  number  of  blanks  is  a better  measure.  The  number  of 
correct  positions  is  influenced  to  an  indeterminant  extent  by 
guessing;  if  subjects  who  are  unable  to  complete  an  operation 
prior  to  deadline  use  the  partial  information  to  guess,  then 
guessing  probabilities  could  vary  markedly.  At  the  same  time 
one  is  assuming  that  in  all  positions  where  the  subjects  respond 
the  same  processes  are  involved  and,  more  importantly,  that 
these  processes  have  all  been  completed.  On  the  other  hand, 
if  a position  is  blank  then  it  is  reasonably  safe  to  conclude 
that  minimal,  if  any,  operations  have  been  applied,  or  that 
subjects  have  not  completed  enough  computation  to  even  make  a 
guess.  Thus  in  a deadline  situation,  the  number  of  blanks 
should  be  more  reflective  of  system  stress.  This  is  evidenced 
in  the  better  fit  of  blanks  (Figure  7)  to  predictions. 

Letter  Recall 

A number  of  different  paradigms  and  theories  lead  to  con- 
sistent predictions  about  performance  in  the  Letter  Recall 
task.  The  memory  span  studies,  the  studies  and  theories  of 
proactive  interference,  and  theoretical  explanations  of  dis- 
placement (e.g.,  Atkinson  and  Shiffrin,  1968)  all  suggest  that 
performance  should  deteriorate  as  the  length  of  the  series  is 
increased.  Thus  we  should  expect  mean  performance  to  decrease 
as  the  number  of  letters  is  increased. 

Figure  8 summarizes  the  performance  on  recall  of  the 
last  five  letters  with  and  without  regard  for  order.  The  series 
recall  measure  was  discarded  when  preliminary  analyses  showed 
it  to  be  influenced  by  a ceiling  effect.  Data  are  presented 
as  a function  of  the  number  of  updates  required,  the  assumption 
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being  that  when  five  letters  are  presented  recall  requires 
only  that  the  letters  be  held  in  memory  and  retrieved  but  when 
six  or  more  letters  are  presented  the  subject  must  manipulate 
or  update  the  information  (e.g.,  dropping  and  adding  letters, 
or  tagging  letters  as  "FORGET"  and  "REMEMBER").  Thus,  the 
important  variable  is  the  number  of  updates  rather  than  the 
number  of  letters  presented,  even  though  one  is  a linear  func- 
tion of  the  other. 

The  functions  look  very  much  as  one  would  expect  from 
predictions,  with  the  curves  reflecting  the  subjects'  increas- 
ing inability  to  control  the  displacement  or  updating  process. 
Analyses  of  variance  for  recall  with  and  without  regard  for 
order  showed  a significant  effect  for  the  number  of  letters 
(F  = 80.32  and  89.82,  respectively;  df  = 5,335;  p < .01). 

Complete  ANOVA  summary  tables  appear  in  Appendix  B. 

Sentence  Recognition 

Two  measures  of  behavior  were  analyzed,  the  accuracy  of 
the  subjects'  "OLD"  or  "NEW"  judgments  and  the  confidence 
rating  given  each  judgment.  Following  the  procedure  of 
Bransford  and  Franks  (1971),  confidence  ratings  were  transformed 
so  that  the  ratings  were  integer  values  on  a scale  from  -5  to 
+5,  excluding  zero.  Figures  9 and  10  summarize  the  findings 
for  these  two  measures.  Since  both  reflected  essentially  the 
same  results,  in-depth  analyses  were  performed  on  only  the 
rescaled  rating  data,  the  measure  of  primary  concern  to  Bransford 
and  Franks. 

The  results  of  the  ANOVA  are  presented  in  Appendix  B, 

Table  B.  Of  primary  interest  in  the  present  context  is  whether 
our  data  exhibit  the  effect  found  by  Bransford  and  Franks;  in 
other  words,  they  found  that  confidence  ratings  on  new-consistent 
sentences  increased  in  value  as  sentence  complexity  increased. 

Our  own  results  indicate  a significant  interaction  among  sen- 
tence complexity,  newness/oldness  and  consistency  (F  = 11.44; 
df  = 2,134;  p < .01)  but  as  shown  in  Figure  10,  confidence 
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Figure  9.  Sentence  Recognition:  Accuracy  of  Old-New  Responses 
for  the  Four  Levels  of  Sentence  Complexity 
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ratings  for  the  new-consistent  condition  do  not  increase  with 
increasing  sentence  complexity.  Other  studies  (e.g..  Parks  and 
Whitten,  1977)  also  fail  to  find  the  Bransford  and  Franks 
effect,  or  find  a diminished  effect,  when  the  subjects  know  the 
nature  of  the  subsequent  recognition  test  beforehand. 

Failure  to  find  the  Bransford  and  Franks  effect  requires 
that  a different  approach  be  used  to  determine  whether  the 
data  are  representative  of  the  abstraction  operation.  Fortunately, 
one  can  make  a good  argument  for  using  the  means  data  in  the 
new-old,  consistent-inconsistent  interaction  table.  The  pattern 
of  performance  in  these  cells  should  vary  according  to  the 
abstraction,  storage  and  search  processes  of  the  individual. 

For  example,  recognition  of  the  old-consistent  sentences  should 
depend  upon  the  ator- ie  and  search  processes  while  false  recog- 
nition of  the  new-inconsistent  sentences  should  depend  upon  the 
abstraction  processes  (i.e.,  matching  of  abstracted  concepts 
from  the  new  sentence  with  previously  abstracted  concepts) . To 
illustrate  how  such  a procedure  might  work  at  the  group  level 
consider  the  following  predictions.  A comparison  of  accuracy 
on  the  consistent-  and  inconsistent-old  sentences  should  show 
higher  accuracy  on  the  consistent-old  sentences  since  the  num- 
ber of  retained  stimulus  dimensions  (i.e.,  event,  syntax,  seman- 
tics) necessary  for  task  performance  is  fewer.  On  the  other  hand 
fot  the  consistent-  and  inconsistent-new  sentences  accuracy 
should  be  higher  for  the  inconsistent-new  sentences  since 
these  sentences  provide  distinctive  syntactic  cues  which  can 
be  used  as  the  basis  of  correct  rejection.  To  the  extent  that 
subjects  abstract  a kernel  representation  of  the  sentences, 
the  trends  should  be  that  the  inconsistent-old  should  be 
difficult  (since  their  representations  would  be  similar  to 
the  encoding  of  the  sentence  when  presented  again) , as  should 
the  consistent-new.  Similarly,  abstraction  would  be  beneficial 
for  the  consistent-old,  since  in  that  case  the  similarity  of 
representations  would  lead  to  a correct  answer.  To  test  these 
predictions  the  appropriate  treatment  interaction  means  were 
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computed  using  the  relative-frequency-correct  data.  This  inter- 
action table,  presented  below,  shows  the  obtained  means  for  the 
old/new  and  consistency  conditions  collapsed  across  complexity: 

CONSISTENT  INCONSISTENT 

OLD  .788  .494 

NEW  .447  .705 

This  table  is  entirely  in  line  with  the  above  predictions, 
showing  highest  accuracy  on  the  consistent-old  and  the  incon- 
sistent-new sentences.  Thus  these  data  provide  support  for 
the  hypothesized  operation  of  abstraction  processes  in  the 
task . 

Sentence  Recall 

This  task  was  included  for  different  primary  purposes  than 
the  other  tasks.  Based  on  our  literature  review,  there  seemed 
to  be  an  apparent  "gap"  in  the  research  concerning  recall  and 
clustering  processes  for  sentences.  And,  although  the  stimuli 
were  derived  partially  from  the  Bransford  and  Franks  (1971) 
work,  the  tasks  developed  here  are  theoretically  and  practically 
different.  Therefore,  a "new"  task  was  developed  as  a poten- 
tial source  of  individual  difference  measures  of  information- 
processing operations. 

For  this  task,  there  are  no  predictions  about  the  nature 
of  the  group  data.  There  are  no  previously  existing  findings 
to  replicate,  since  there  does  not  appear  to  be  a comparable 
task  in  the  experimental  literature.  While  it  is  tempting  to 
make  predictions  using  subjective  cognitive  computations  based 
on  other  paradigms  and  stimuli,  we  firmly  believe  that  the 
differences  in  important  dimensions  between  the  paradigms  and 
stimuli  make  these  comparisons  meaningless.  Further,  the  task 
manipulations  here  were  not  designed  to  produce  any  particular 
group  effect.  Thus  the  task  functions  solely  in  the  develop- 
ment of  the  individual  difference  measures. 
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For  the  sake  of  completeness,  the  group  data  on  this  task 
are  shown  in  Figure  11.  Two  variables  were  used  to  measure 
behavior,  an  item  recall  measure  and  a cluster  recall  measure. 

The  item  recall  procedure  counted  the  number  of  nouns,  verbs, 
and  adjectives  reproduced  in  recall  from  the  original  sentences, 
while  the  cluster  recall  procedure  tallied  the  number  of  con- 
tiguously recalled  ideas  or  attributes  from  a particular  topic. 
Mean  performance  (percentage  correct)  on  the  two  measures  for 
each  day  of  testing  as  a function  of  the  number  of  attributes 
(4,  5,  and  6)  is  shown  in  Figure  11.  This  figure  indicates 
no  clear  pattern  of  results  for  either  measure,  especially  with 
respect  to  consistency  across  days.  Tables  B-7  & 8 in  Appendix 
B present  the  analyses  of  variance  summary  tables. 

Group  Measures  Summary 

Table  2 provides  a general  summary  of  the  findings  with 
the  group  measures  data.  The  redults  indicate  that,  where 
applicable,  the  major  group  effects  were  replicated  in  almost 
every  paradigm.  For  two  of  the  tasks.  Sentence  Recall  and 
Physical  Match,  replication  was  not  an  issue.  The  Sentence 
Recall  task  was  a new  task  of  our  design  so  there  was  no  previous 
literature  of  results  to  replicate.  The  Physical  Match  task 
involved  a single  treatment  condition,  so  this  task  could  only 
be  compared  to  previous  results  at  a general  level.  Of  the 
remaining  tasks  only  the  Sentence  Recognition  task  failed  to 
replicate  previous  findings,  an  outcome  which  appears  to  result 
from  a change  in  the  subject's  knowledge  about  the  test  that 
was  necessary  for  our  purposes.  Even  in  this  case,  the  litera- 
ture provides  some  support  for  the  obtained  results  under 
conditions  of  prior  knowledge  about  the  test. 

Therefore,  we  are  confident  that  these  tasks,  as  implemen- 
ted, are  "solid"  paradigms.  They  produce  phenomena  that  are 
consistent  with  previous  findings  or  are  demonstrably  capable 
of  straightforward  interpretations  at  the  group  level.  As 
such,  these  results  add  additional  logical  support  for  the 
interpretation  of  task  performance  as  representing  the  operations 
shown  in  Table  1. 
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Overview.  The  value  of  the  tasks  as  part  of  an  information 
processing  test  battery  depends  primarily  on  the  usefulness  of 
measures  derived  from  the  tasks  as  indices  of  individual 
differences.  The  group  measures  discussed  earlier  serve  mainly 
to  affirm  that  the  present  forms  of  the  tasks  can  indeed  pro- 
duce the  behavior  extant  in  the  literature  and  to  substantiate 
our  judgments  about  the  theoretical  processes  involved.  The 
individual  measures  discussed  in  this  section  serve  to  illus- 
trate the  ability  of  the  tasks  to  detect  individual  differences 
in  an  interpretable  manner.  The  primary  considerations  are 
the  reliability  of  task  effects,  the  presence  of  practice 
effects,  and  some  index  of  construct  validity.  These  mea- 
surement issues  are  discussed  below. 

For  the  examination  of  individual  performance  and  construct 
validity,  a set  of  59  indices  was  generated  from  the  data. 

Most  of  these  indices  were  derived  to  capture  the  essential 
features  of  the  basic  phenomena  in  each  paradigm;  as  will  be 
discussed  below  in  the  context  of  construct  validity,  each  of 
these  indices  is  interpretable  in  terms  of  the  information 
processing  "operations"  vocabulary.  The  remaining  indices 
were  derived  as  summary  measures  in  a particular  task  or  task 
condition;  these  indices  were  thought  to  be  more  general  indi- 
cators of  a particular  subject's  operational  capabilities  and 
hence  might  be  valuable  as  individual  difference  parameters. 

A few  indices  were  added  to  the  set  for  tasks  previously  studied 
in  the  course  of  this  research  project;  these  consist  of  separate 
Day  1 and  Day  2 measures,  and  were  included  primarily  to  facili- 
tate comparison  with  previous  data.  These  59  indices  are  as 
follows . 

Letter  Recall.  For  each  of  the  three  principal 
scores  (percentage  of  letters  correct  in  order, 
percentage  of  letters  correct  without  regard  to 
order,  and  percentage  of  letters  correct  with 
respect  to  the  entire  series) , three  indices 
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were  computed:  the  slope  and  intercept 
of  the  best-fitting  linear  function*  relating 
percentage  correct  and  number  required  updates, 
and  a summary  percentage  correct  (collapsed 
across  number  of  updates) . 


Mental  Addition.  For  the  two  principal  scores 
(number  of  blanks  and  number  correct) , five 
indices  were  computed:  the  slope  and  intercept 
of  the  best-fitting  linear  function  relating 
the  principal  score  and  the  "memory  load",  the 
summary  mean  scores  of  the  3+2  problems  and 
the  3+3  problems  collapsed  across  number  of 
carries;  and  a summary  mean  performance  score, 
collapsed  across  all  problems. 

Sentence  Recall.  For  the  cluster  measure,  three 
indices  were  derived:  the  number  of  clusters 
recalled  correctly  weighted  by  cluster  size  and 
averaged  across  the  number  of  sentence  attributes; 
a "maximum  correct  score"  which  indicated  the 
largest  (average)  number  of  clusters  recalled 
correctly  for  any  number  of  sentence  attributes; 
and  the  attribute  level  where  this  maximum  score 
occurred.  For  example,  suppose  a subject  correctly 
recalled  three  out  of  four  4-attribute  clusters, 
two  out  of  four  5-attribute  clusters,  and  one  out 
of  four  6-attribute  clusters.  The  mean  cluster 
score  would  be  3 (i.e.,  3 x 4/4)  + 2.5  (i.e.,  2 x 
5/4)  + 1.5  (i.e.,  1 x 6/4)  = 7/3  or  2.3.  The 
maximum  correct  score  would  be  3.0,  and  the  attri- 
bute level  score  would  be  4. 


These  functions  do  not  include  the  five-letter  (no  update) 
conditions  because  of  the  marked  non-linearity  of  this  point 
from  the  remaining  conditions. 
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Sentence  Recognition.  A total  of  five  indices  were 
derived,  four  of  which  consisted  of  the  mean  ratings 
for  each  of  the  sentence  types  (old-consistent, 
old- inconsistent,  new-consistent,  and  new-inconsistent) . 
The  fifth  index  was  a summary  measure  (mean  per- 
centage of  errors  collapsed  across  sentence  types)  of 
subjects’  accuracy. 

Letter  Rotation.  Two  basic  indices  were  derived, 
namely  the  slope  and  intercept  of  the  best-fitting 
linear  function  relating  time  per  item  and  number  of 
degrees  of  rotation.  Since  it  is  unclear  whether 
including  or  excluding  errors  is  the  more  appropriate 
measure,  both  have  been  included  in  the  set.  Also, 
since  previous  data  are  available  for  these  indices, 
both  Day  1 and  Day  2 scores  are  included. 

Physical  Match.  The  single  index  derived  for  this  task 
is  the  mean  time  per  item.  Again,  since  it  is  unclear 
whether  errors  should  be  included  or  excluded,  both 
have  been  retained,  as  have  Day  1 and  Day  2 scores. 

Set  Membership.  The  two  indices  derived  for  this  task 
are  the  slope  and  intercept  of  the  best-fitting 
linear  function  relating  time  per  item  to  memory  set 
size.  For  the  reasons  mentioned  above,  both  including 
and  excluding  errors  have  been  included,  as  have  Day 
1 and  Day  2 scores. 

Scan  and  Search.  The  indices  derived  for  this  task  are 
identical  to  the  Set  Membership  indices  (with  "target 
set  size"  replacing  "memory  set  size")  for  both  the 
clear  and  degraded  stimulus  conditions.  However,  it 
was  not  considered  necessary  to  include  both  Day  1 
and  Day  2 scores  individually,  since  the  direct  com- 
parison with  previous  data  would  be  questionable. 

Table  3 presents  detailed  descriptive  information  for 
each  of  these  59  individual  parameters.  This  information 
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Sentence  Sentence 

Recognition  Recall  Mental  Addition  Letter  Recall 


Table  3 


Descriptive  Statistics  for 

the  Task 

Parameters 

Std. 

k Parameter 

Mew 

Dev. 

Min 

With  Order,  Intercept 

0.75 

0.15 

0.44 

With  Order,  Slope 

-0.03 

0.03 

-0.12 

Without  Order,  Intercept 

0.86 

0.09 

0.56 

Without  Order,  Slope 

-0.02 

0.02 

-0.07 

Within  Series,  Intercept 

0.90 

0.07 

0.62 

Within  Series,  Slope 

-0.00 

0.01 

-0.04 

With  Order,  Metui 

0.67 

0.09 

0.45 

Without  Order,  Mean 

0.81 

0.07 

0.57 

Within  Series,  Mean 

0.89 

0.07 

0.65 

Blanks , Intercept 

-0.34 

0.19 

-0.74 

Blanks,  Slope 

0.08 

0.04 

-0.00 

Blanks,  3+2 

0.12 

0.09 

0.00 

Blanks,  3+3 

0.19 

0.12 

0.00 

Blanks,  Overall  Mean 

0.16 

0.10 

0.00 

Correct , Intercept 

1.37 

0.21 

0.81 

Correct,  Slope 

-0.10 

0.04 

-0.17 

Correct,  3+2 

0.77 

0.10 

0.49 

Correct,  3+3 

0.68 

0.11 

0.42 

Correct , Overall  Mean 

0.73 

0.10 

0.49 

Mean  Cluster 

1.38 

0.80 

0.00 

Max.  Cluster 

2.00 

1.04 

0.00 

Cluster  Level 

4.95 

0.85 

4.00 

Mew  Error 

0.40 

0.07 

0.27 

Rating  by  Condition,  New/Incon. 

-1.60 

0.97 

-3.25 

Rating  by  Condition,  Old/Incon. 

-0.18 

1.16 

-3.00 

Rating  by  Condition,  New/Consist. 

0.59 

1.16 

-1.75 

Rating  by  Condition,  Old/Consist. 

2.45 

1.04 

-1.42 
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Max. 

1.06 

0.03 

1.02 

0.03 

1.00 

0.03 

0.91 

0.93 

0.98 

0.03 

0.17 

0.35 

0.47 

0.39 

1.78 

-0.02 

0.93 

0.88 

0.90 

2.92 

3.75 

6.00 

0.55 

1.69 

3.50 

3.00 

4.83 


Table  3 (Continued) 

Descriptive  Statistics  for  the  Task  Parameters 


Task  Parameter 

Mean 

Std. 

Dev. 

Min. 

Inc . Errors , 

Intercept , Day  1 

1.35 

0.51 

0.63 

Inc.  Errors , 

Slope,  Day  1 

0.01 

0.01 

0.00 

6 

•H 

Inc . Errors , 

Intercept,  Day  2 

1.03 

0.34 

0.49 

4J 

Inc.  Errors, 

Slope,  Day  2 

0.01 

0.00 

0.00 

s 

Exc.  Errors, 

Intercept,  Day  1 

1.60 

1.02 

0.57 

u 

Q) 

Exc . Errors , 

Slope,  Day  1 

0.01 

0.01 

0.00 

■P 

Exc . Errors , 

Intercept,  Day  2 

1.13 

0.62 

0.31 

Exc.  Errors, 

Slope , Day  2 

0.01 

0.00 

0.00 

Inc.  Errors, 

Day  1 

0.71 

0.11 

0.50 

£ 

Inc . Errors , 

Day  2 

0.58 

0 .08 

0.42 

U 

4J 

<G 

Exc . Errors , 

Day  1 

0.71 

0.12 

0.50 

X 

Exc . Errors , 

Day  2 

0.58 

0 .08 

0 .42 

Inc . Errors , 

Intercept,  Day  1 

0.50 

0.10 

0.27 

Inc.  Errors, 

Slope,  Day  1 

0.06 

0.02 

0.01 

04 

•H 

Inc . Errors , 

Intercept,  Day  2 

0.42 

0.07 

0.22 

.C 

w 

u 

Inc.  Errors, 

Slope , Day  2 

0.06 

0.02 

0.02 

Exc.  Errors, 

Intercept,  Day  1 

0.49 

0.10 

0.25 

Exc . Errors , 

Slope,  Day  1 

0.06 

0.03 

0.02 

4J 

0) 

w 

Exc.  Errors, 

Intercept,  Day  2 

0.41 

0.07 

0.22 

Exc . Errors , 

Slope , Day  2 

0.07 

0.03 

0.03 

Inc.  Errors, 

Clear,  Intercept 

0.19 

0.12 

-0.25 

Inc.  Errors, 

Clear,  Slope 

0.28 

0.09 

0.12 

x: 

Exc.  Errors, 

Clear , Intercept 

0.12 

0.11 

-0.27 

o 

M 

(TJ 

Exc.  Errors, 

Clear,  Slope 

0.33 

0.09 

0.18 

a 

Inc.  Errors, 

Clear,  Overall  Mean 

0.84 

0.14 

0.57 

•0 

§ 

Exc.  Errors, 

Clear,  Overall  Mean 

0.91 

0.14 

0.60 

§ 

Inc . Errors , 

Degraded , Intercept 

0.30 

0.13 

-0.01 

8 

Inc.  Errors, 

Degraded , Slope 

0.26 

0.09 

0.11 

Exc.  Errors, 

Degraded,  Intercept 

0.19 

0.17 

-0.33 

Exc.  Errors, 

Degraded,  Slope 

0.36 

0.13 

0.16 

Inc . Errors , 

Degraded,  Overall  Mean 

0.91 

0.16 

0.63 

Exc.  Errors, 

Degraded,  Overall  Mean 

1.04 

0.20 

0.72 
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Max. 

2.83 

0.04 

2.41 

0.02 

6.40 

0.04 

4.21 
0.03 

0 .98 
0.75 
1.00 
0.75 

0.71 

0.12 

0.59 

0.13 

0.69 

0.15 

0.59 

0.16 

0.51 

0.54 

0.37 

0.56 

1.21 
1.27 
0.71 
0.54 
0.63 
0.86 
1.36 
1.68 


includes  the  mean,  standard  deviation,  and  maximum  and  minimum 
scores.  These  data  are  included  for  comparison  with  previous 
and  future  replications  and  for  future  use  as  an  information- 
processing data  base. 

It  should  be  noted  that  there  is  an  apparent  "scale" 
problem  associated  with  most  of  the  slope  indices.  For  example, 
consider  the  Letter  Rotation  slopes.  Although  the  interpreta- 
tion of  the  absolute  time  value  of  the  slope  (i.e.,  seconds 
per  degree  of  rotation)  is  in  line  with  previous  research,  the 
descriptive  data  in  the  table  are  truncated.  In  fact  however 
(as  will  be  shown  in  succeeding  sections) , there  is  sufficient 
variance  in  subjects'  scores  to  produce  substantial  correlations. 

It  was  perhaps  inappropriate  to  present  the  slope  score  to  the 
nearest  hundredth  of  a second  (rather  than  in  msec) ; however, 
we  chose  to  avoid  the  complication  of  changing  the  units  of 
measurement  both  within  this  table  and  between  this  table  and 
previous  presentations  of  results. 

Reliabilities  and  Practice  Effects.  Test-retest  reliabilities 
for  selected  indices  are  presented  in  Table  4.  Omitted  from 
this  table  are  the  "summary"  indices.  In  line  with  our  general 
approach,  we  would  probably  not  ultimately  recommend  such  indices 
be  incorporated  in  an  information  processing  assessment  battery. 
Since  our  objective  is  to  design  a test  battery  which  measures 
individual  differences  in  cognitive  operations,  indices  which 
summarize  a subject's  synthesis  of  several  operations  (or  more 
generally,  task  performance  collapsed  across  different  conditions) 
are,  in  our  judgment,  relatively  less  desirable. 

It  should  be  kept  in  mind  that  test-retest  correlations 
are  not  commonly  used  as  a reliability  criterion.  Typically, 
split-half,  odd-even,  or  alternate-form  correlations  are 
reported.*  Also,  it  should  be  noted  that  we  were  unable  to 


These  calculations  were  precluded  for  the  tasks  in  this  study 
due  primarily  to  the  group-administration  format. 
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Table  4 


Test-Retest  Correlations  for  the  Task  Parameters 


Task  Parameter 


Test-Retest 

Correlation 


1 


With  Order,  Intercept 

.51 

With  Order,  Slope 

.21 

Without  Order,  Intercept 

.49 

Without  Order,  Slope 

.19 

Within  Series,  Intercept 

.41 

Within  Series,  Slope 

.24 

Blanks,  Intercept 

.39 

Blanks,  Slope 

.61 

Correct,  Intercept 

.14 

Correct,  Slope 

.35 

Mean  Cluster 

.37 

Max  Cluster 

.41 

Cluster  Level 

.12 

Errors 

.18 

New-Consistent 

.14 

New- Inconsistent 

.27 

Old-Consistent 

.30 

Old- Inconsistent 

.31 

Inc.  Errors,  Intercept 

.72 

Inc.  Errors,  Slope 

.53 

Exc.  Errors,  Intercept 

.64 

Exc.  Errors,  Slope 

.75 

Inc.  Errors,  Mean 

.85 

Exc.  Errors,  Mean 

in 

CD 

• 

Table  4 (Continued) 


r 


i 


*. 

f* 


Test-Retest  Correlations  for  the 


Task 

Parameter 

% 

Inc.  Errors 

w 

4->  J-J 

Inc.  Errors 

Exc.  Errors 

1 

Exc.  Errors 

Inc.  Errors 

•g 

Inc.  Errors 

| 

Inc.  Errors 

V, 

Inc.  Errors 

3 

Exc.  Errors 

1 

Exc.  Errors 

Exc.  Errors 

Exc.  Errors 

Intercept 

Slope 

Intercept 

Slope 

Clear,  Intercept 
Clear,  Slope 
Degraded,  Intercept 
Degraded,  Slope 
Clear,  Intercept 
Clear,  Slope 
Degraded,  Intercept 
Degraded,  Slope 


l. 


4m 
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Task  Parameters 

Test-Retest 

Correlation 

.66 

.81 

.66 

.79 

.77 

.76 

.70 

.71 

.63 

.68 

.55 

.60 


find  in  the  literature  any  reported  reliability  measures  for 
the  tasks  or  task  parameters  investigated  in  this  study. 

The  reliabilities  shown  in  Table  4 range  from  r = .12 
(Sentence  Recall,  cluster  level)  to  r = .85  (Physical  Match, 
time  per  item) . Four  of  the  tasks — Physical  Match,  Set  Member- 
ship, Letter  Rotation,  and  Scan  and  Search — have  indices  which 
show  "good"  (in  our  judgment)  reliabilities  of  r = .70  or  above. 
We  believe  that  these  reliabilities  are  acceptable,  even  from 
the  perspective  of  standard  test  development.  Interestingly, 
these  are  the  tasks  that  use  a (derived)  time  metric,  as  opposed 
to  the  other  tasks  that  use  a percent-correct  metric.  These 
other  tasks — Sentence  Recall,  Mental  Addition,  Letter  Recall, 
and  Sentence  Recognition — while  occasionally  showing  an  index 
with  a moderately  high  reliability  (r  = .40  or  above),  are 
generally  disappointing.  Clearly,  from  a test-development 
standpoint,  further  work  must  be  done  before  these  tasks  and 
measures  would  be  recommended  for  inclusion  in  a test  battery. 
This  work  might  include  the  generation  of  a different  set  of 
indices,  longer  tests,  or  modifications  of  testing  conditions 
(e.g.,  different  treatments,  instructional  variations,  etc.). 

A further  consideration  is  that  these  tasks  seem  to  be  more 
sensitive  to  shifts  in  subjects'  strategies  between  administra- 
tions than  are  the  other  tasks.  Indeed,  the  possibility  that 
shifts  did  occur  was  partially  confirmed  during  informal  con- 
versations with  the  subjects. 

In  light  of  the  above  discussion  of  reliability,  the  inter- 
pretation of  practice  effects  is  perhaps  superfluous.  The  data 
showed  significant  practice  (Day)  effects  for  all  tasks  except 
Sentence  Recall  (see  Appendix  B for  statistical  confirmation 
and  critical  values) . These  practice  effects  resulted  from  an 
increase  in  accuracy  or  a decrease  in  required  processing  time. 
Generalizations  about  the  practice  effects  must  also  be  tempered 
because  of  the  significant  interactions  with  other  treatment 
conditions.  Implications  of  these  practice  effects  for  test 
development  and/or  test  battery  implementation  are  twofold: 
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first,  as  implied  by  the  reliability  discussion  above,  more 
research  must  be  done  to  eliminate  or  regulate  "unwanted" 

(i.e.,  strategy  shift)  practice  effects;  and  second,  care  must 
be  taken  in  the  design  of  an  experimental  implementation.  That 
is,  proper  control  groups  must  be  employed,  proper  statistical 
procedures  (e.g.,  analyses  of  covariance)  must  be  used,  and 
so  on. 

Construct  Validity 

Overview.  The  concept  of  construct  validity  is  relatively 
new  in  experimental  psychology.  While  the  concept  is  of  critical 
importance  to  any  research  concerned  with  test  development, 
individual  differences,  and  performance  assessment,  there  is 
no  formal  statistical  procedure  for  its  evaluation.  Typically 
suggested  procedures  are  general  and  involve  procedures  like: 

(1)  specification  of  observables  relevant  to  the  construct, 

(2)  determination  of  extent  to  which  observables  measure  the 
same  thing,  and  (3)  determination  of  the  extent  to  which  supposed 
measures  of  the  construct  produce  results  predicted  from  accep- 
ted theoretical  hypotheses  about  the  construct  (Nunnally;  1978, 
p.  98 ) . 

The  approach  used  here  follows  fairly  closely  the  proce- 
dures outlined  by  Nunnally,  with  some  allowances  for  the  par- 
ticular situation  and  operations  involved.  A sample  from  the 
domain  of  observables  has  been  specified;  these  are  the  59 
individual  parameters  listed  in  Table  5.  The  table  also  shows 
the  major  operations  involved  (derived  primarily  from  Table  1) 
and  the  psychological  interpretations  for  the  measures.  The 
extent  to  which  the  observed  data  from  tasks  comply  with 
theoretical  predictions  was  demonstrated  in  the  foregoing  dis- 
cussion of  group  measures.  Thus  all  that  remains  is  to  demon- 
strate the  ability  of  the  different  observables  to  measure  the 
same  construct  (operation) . The  redundancies  in  Table  5 were 
included  for  just  this  purpose. 

An  inspection  of  Table  5 (or  even  more  clearly,  Table  1) 
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TABLE  S. 

Operations  and  Interpretations  of  Task  Parameters 

Task  Parameter 

Interpretation 

1.  Letter  Recall:  Operations- storage  and  retrieval  (short  term  retention) 

• with  order  • slope 

• intercept 

• mean 

rate  of  item  and  order  information  loss  with  each  update 

protected  item  and  order  information  capabilities  in  absence  of  updates 

summarized  item  and  order  retention  capabilities 

• without  order  •slope 

• intercept 
•mean 

rate  of  item  information  loss  with  each  update 

projected  item  information  capabilities  in  absence  of  updates 

summarized  item  retention  capabilities 

• within  series  •slope 

• intercept 

• mean 

rate  of  scries  information  loss  with  each  update 

projected  series  information  retention  capabilities  in  absence  of  updates 
summarized  retention  from  scries 

2.  Mental  Addition:  Operations  transformation,  storage  and  retrieval  (short  term  retention) 

• number  blanks  • slope 

• intercept 
•mean  3+2  j 

mean  3+3  , 

mean  overall 

rate  of  information  loss  with  increasing  memory  load 

unclear 

| summarized  capabilities  for  retention  and  transformation  operations 

• number  correct  • slope 

• intercept 

• mean  3+2  j 
mean  3+3 
mean  overall  ] 

rate  of  information  loss  with  increasing  memory  load 

unclear 

t 

k summarized  capabilities  for  retention  and  transformation  operations 

1 

3.  Sentence  Recall:  Orations  abstraction,  storage,  retrieval 

• cluster  recall  • mean  complete- 
ly corrcc  capabilities  for  abstraction  and  retention 

• maximum  com- 
pletely correct  capabilities  for  abstraction  and  retention 

• level  with  maxi- 

mum complete- 
ly correct  capabilities  for  abstraction  and  retention 


4.  Sentence  Recognition:  Operations  abstraction,  storage,  search 


• mean  error 

• rating  old /inconsistent 

• rating  old/consistent 

• rating  new/inconsistent 

• rating  new/consistent 


abstraction  and  retention  capabilities 
retention  of  details  for  old  sentences 
retention  and  abstraction  for  old  sentences 
abstraction  of  concepts 
abstraction  of  concepts  and  details 


5.  Letter  Rotation:  Operations  transformation,  comparison 

• slope  (with  errors) 

without  errors)  time  required  to  transform  tier  degree  of  rotation 

• intercept  (with  errors, 

without  errors)  time  required  to  make  a comparison 


6.  Physical  Match:  Operation  comparison 

• mean  (with  errors,  without  errors)  time  required  to  make  a comparison 


7.  Set  Membership:  Operations-storage,  search 

• slope  (with  errors. 

without  errors)  time  required  to  search  storage  per  item  in  storage 

• intercept  (with  errors. 

without  errors)  time  required  to  maintain  access  to  memory  set  and  encode  probe 


8.  Scan  and  Search:  Operations— Storage,  search,  encoding 

• clear  stimuli  • slope  (with  errors, 

without  errors)  time  required  to  search  per  item  in  target  set 

• intercept  (with 
errors,  without 

onors)  time  required  to  maintain  access  to  target  and  encode  stimuli 

•mean  (with  errors. 

without  errors)  summarized  processing  time  for  operations 

• degraded  ‘slope  (with  errors, 

stimuli  without  errors)  time  required  to  search  per  item  in  target  set 

• intercept  (with  errors, 

without  errors)  time  required  to  maintain  access  to  target  and  encode  stimuli 

• mean  (with  errors. 

without  errors)  summarized  processing  time  for  operations 
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reveals  that,  with  the  exception  of  encoding,  recoding,  and 
decision-response,  each  operation  is  involved  in  at  least  two 
tasks.  If  the  "major"  operations  are  indeed  significant  factors 
in  task  performance,  then  one  might  hope  that  measures  from 
tasks  sharing  the  same  operations  would  be  correlated.  At  the 
risk  of  appearing  pedantic,  it  must  be  repeated  that  such  an 
expectation  depends  upon  a chain  of  inferences,  each  link  of 
which  relies  primarily  upon  interpretation  and  judgment. 

Table  1 is  essentially  a series  of  theoretical  hypotheses  about 
operations.  As  such,  these  hypotheses  reflect  the  authors' 
particular  (or  peculiar)  theoretical  biases,  vocabulary,  and 
task  analyses.  The  next  judgment  concerns  the  interpretation 
of  the  individual  tasks'  group  effects  as  more  or  less  suppor- 
tive of  the  underlying  operational  descriptions.  For  the  most 
part,  we  have  considered  "phenomenon  replicability"  as  presump- 
tive evidence  for  these  interpretations;  confidence  has  been 
increased  not  only  from  the  present  results  but  also  from  the 
results  of  other  investigators  who  have  performed  empirically- 
based  converging  operations.  The  next  judgment,  represented 
in  Table  5,  is  the  designation  of  measures  as  reflecting  one 
or  more  operations.  For  the  many  measures  that  adequately 
represent  task  performance,  the  judgment  was  made  as  to  the 
relevance  of  each  to  the  operational  construct.  The  final 
step  in  the  chain  of  inferences  is  the  correlational  hypothesis 
that  two  measures  sharing  the  same  operation  will  be  statis- 
tically related.  Of  course,  if  each  parameter  was  hypothesized 
to  measure  only  one  operation,  the  evidence  would  be  direct. 
However,  the  evidence  becomes  shakier  when  both  parameters 
measure  more  than  one  operation.  Without  assumptions  concerning 
relative  weights  or  correlations  among  the  operations  themselves, 
the  interpretation  of  the  observed  correlations  is  indirect. 
Nonetheless,  using  Table  5 as  a guide,  we  expect  that  parameters 
supposedly  measuring  the  same  process  (e.g.,  physical  match- 
mean  and  letter  rotation-intercept)  would  be  correlated.  Thus 
using  Table  5 a number  of  "expected"  significant  correlations 
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can  be  generated.  Table  6 presents  the  obtained  correlations. 
Their  correspondence  to  selected  predictions  is  discussed  below. 

Storage/Retrieval  Operations.  Storage/retrieval  operations 
should  show  up  as  significant  correlations  in  comparing  letter 
recall,  mental  addition,  and  sentence  recall.  Sentence  recall 
produced  significant  correlations  with  all  the  letter  recall 
means  and  some  of  the  intercept  measures  for  letter  recall. 

With  mental  addition,  sentence  recall  correlated  significantly 
with  the  means  for  blank  positions.  For  letter  recall  and  mental 
addition,  almost  all  of  the  correlations  with  the  letter  recall 
intercept  measures  are  significant  as  well  as  the  correlations 
between  mental  addition  means  and  letter  recall  means.  Thus, 
there  is  good  inferential  support  for  the  construct  validity 
of  the  storage/retrieval  operations  and  measures  reflecting 
them. 

Comparison  Operation.  The  comparison  operation  should 
produce  significant  correlations  between  physical  match  and 
letter  rotation.  Indeed  Table  6 shows  significant  or  marginally 
significant  correlations  in  many  of  the  possible  comparisons 
of  physical  match  and  letter  rotation,  the  major  exceptions 
being  those  cells  involving  the  letter  rotation  intercept 
excluding  errors  on  Day  1.  Again,  there  is  good  supDort  for 
the  construct  validity  of  the  comparison  operation. 

Abstraction  Operation.  By  investigating  the  correlation 
between  sentence  recall  and  sentence  recognition,  we  can  get 
some  idea  of  the  construct  validity  for  abstraction.  Unfor- 
tunately, this  measure  is  contaminated  by  confounding  with 
storage  operations.  Disregarding  the  cluster  level  score,  we 
again  find  some  significant  correlations. 

Search  Operation.  The  search  operation  should  produce 
significant  correlations  between  set  membership  and  scan  and 
search.  Again,  the  storage  operation  is  a confounding.  The 
majority  of  correlations  here  are  significant.  That  a similar 
effect  is  not  seen  when  we  compare  set  membership  and  letter 
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recall  or  mental  addition,  or  in  comparing  scan  and  search  and 
letter  recall,  suggests  that  more  than  simply  storage  effects 
are  operating  here. 

Implications  for  Test  Battery 

From  the  above  results,  certain  tasks  stand  out  as  strong 
candidates  for  inclusion  in  an  information  processing  performance 
test  battery.  These  tasks  are  the  Physical  Match,  Scan  and 
Search,  Set  Membership,  and  Letter  Rotation  tasks.  These  four 
tasks  replicated  published  findings,  showed  good  reliability, 
and  appeared  to  possess  construct  validity.  Further,  the  tasks 
are  easy  to  administer,  eacy  to  score,  and  alternate  forms 
can  be  easily  constructed.  Recommendations  are  that  these 
tasks  be  given  strong  consideration  for  inclusion  in  any  appli- 
cation of  the  battery. 

The  Letter  Recall  task  and  Mental  Addition  task  comprise 
a second  group  of  more  marginal  candidates.  These  tasks  pro- 
duced significant  effects  consistent  with  expectations  and 
appeared  to  have  some  degree  of  construct  validity  but  reli- 
ability was  moderate  to  low.  The  tasks  are  easy  to  score  and 
alternate  forms  are  easily  constructed.  However,  administration 
is  more  difficult  than  for  the  tasks  discussed  just  previously. 
Both  tasks  require  good  mechanisms  for  timing  stimulus  presen- 
tation and  intervals  consistently.  Thus  while  the  tasks  are 
innovative  approaches  to  measuring  information  processing  opera- 
tions, they  are  also  flawed.  Recommendations  are  that  the  tasks 
be  included  in  any  application  of  the  battery  but  that  attempts 
be  made  to  improve  them. 

The  final,  most  marginal  candidates  for  the  test  battery 
are  the  Sentence  Recall  and  Sentence  Recognition  tasks.  The 
Sentence  Recognition  task  produced  significant  treatment  effects 
but  failed  to  replicate  the  desired  finding.  The  task  produced 
very  low  reliability  and  alternate  forms  are  difficult  to  con- 
struct. The  Sentence  Recall  task  produced  significant  treat- 
ment effects  but  showed  low  to  moderate  reliability  and  was  also 
moderately  difficult  to  construct.  The  difficulty  in  construction 
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with  these  tasks  stems  from  a difficulty  in  controlling  extra- 
neous variables  such  as  sentence  complexity,  sentence  length, 
vocabulary,  and  any  other  variable  that  might  influence  subject 
strategies  from  sentence  to  sentence.  The  tasks  also  are 
moderately  difficult  to  administer,  requiring  moderate  consis- 
tency in  the  timing  of  events.  Construct  validity  appeared 
to  be  good,  but  scoring,  especially  for  the  Sentence  Recall 
task,  is  cimplex  and  time  consuming.  Recommendations  are  that 
these  tasks  be  replaced  with  other  tasks  designed  to  converge 
on  the  abstraction  operation. 
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INTRODUCTION 

Welcome  to  the  American  Institutes  for  Research.  The 
research  you  are  about  to  take  part  in  is  one  phase  of  a larger 
project  designed  to  help  understand  basic  human  information 
processing  capabilities  and  limitations. 

The  results  of  this  project  will  be  used  to  improve 
educational  and  vocational  guidance  programs.  The  project 
will,  for  example,  contribute  to  the  matching  of  individual 
qualifications  and  characteristics  as  needed  for  specific  jobs 
and  to  the  development  of  training  programs  for  various  occu- 
pations and  professions. 

Your  participation  in  this  project  will  require  attendance 
at  two  sessions,  consisting  of  approximately  three  hours  each 
session.  At  the  end  of  the  second  session  you  will  be  paid 
$30  for  your  participation. 

During  each  session  you  will  be  asked  to  complete  a series 
of  simple  tasks.  These  tasks  involve  simple  matching  and 
memory  decisions,  and  do  not  test  your  knowledge  of  general 
information,  your  intelligence,  nor  your  personality. 

Every  effort  has  been  undertaken  to  insure  your  privacy 
with  respect  to  performance  on  these  tasks.  Your  name  will 
never  be  associated  with  the  scores.  Before  the  tasks  are 
scored  your  responses  will  be  assigned  a number  and  that  number 
will  be  used  to  refer  to  the  individual  task  performance.  At 
no  point  will  your  name  be  associated  with  the  assigned  number. 
For  our  ease  in  handling  your  responses,  we  would  like  you  to 
make  up  a pseudonym  that  you  can  write  wherever  the  response 
sheets  say  "NAME" . Please  use  the  same  name  throughout  every 
task  for  both  days.  Do  not  use  your  real  name. 

NAME  (pseudonym)  , sex:  male 

female 


A 
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You  should  be  able  to  complete  today's  tasks  in  about 
three  hours.  There  will  be  a break  about  halfway  through  the 
tasks.  Remember  you  will  be  paid  only  if  you  complete  the 
tasks  for  both  days.  Any  questions? 

Read  with  me  through  the  following  description  of  the  first 
task  and  ask  any  questions  you  may  have. 


70 


INSTRUCTIONS  FOR  THE  LETTER  RECALL  TASK 


In  this  task  we  are  interested  in  your  ability  to  remem- 
ber letters  from  a series  of  letters.  We  shall  read  a series 
of  letters  to  you,  one  letter  at  a time.  At  the  end  of  that 
series  we  shall  ask  you  to  recall  the  last  five  letters  in 
the  order  in  which  they  occurred  in  the  series. 

Each  of  the  series  we  shall  read  will  contain  a varying 
number  of  letters.  Most  of  the  time  there  will  be  more  than 
five  letters  in  the  series.  You  will  not  know  beforehand 
the  number  of  letters  in  any  particular  series.  At  the  end 
of  the  series  we  will  say  the  word  "RECALL";  you  are  to  then 
recall  only  the  last  five  letters  of  the  series.  We  want  you 
to  try  to  remember  the  order  in  which  the  last  five  letters 
were  presented.  You  will  notice  that  on  the  response  sheet 
there  are  five  blanks  for  each  series.  Write  your  five  letters 
in  these  five  blanks.  For  example,  if  we  read  the  following 
letters  to  you  "D  0 Y R L C E" , in  that  order,  your  recall 
in  the  blanks  should  ideally  look  like 

Y R L C E. 

If  you  forget  some  letters  you  should  leave  those  blanks  empty. 
For  example: 

Y R _ C E 

If  you  cannot  remember  the  position  of  a letter  in  the  five 
letters,  guess,  but  try  to  be  as  accurate  as  you  can  in  recalling 
the  last  five  letters  and  their  positions. 

Any  questions? 

We  shall  now  give  you  some  practice. 


STOP. 
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INSTRUCTIONS  FOR  PHYSICAL  MATCH  TASK 


l 

T 

4* 


4. 


i 


L 

I. 

li 

i; 

L 

r 

L 

r 

n 

1 

I 


In  this  task  we  are  interested  in  how  accurately  and 
rapidly  you  can  make  a series  of  simple  judgments.  We  shall 
be  concerned  with  your  ability  to  decide  whether  two  letters 
are  the  same  or  different.  On  the  task  sheet,  you  will 
see  columns  of  pairs  of  letters.  If  the  two  letters  in  a 
pair  are  the  same  put  a check  mark  in  the  space  to  the  left 
of  the  letters;  if  the  two  letters  are  different  put  a check 
mark  in  the  space  to  the  right  of  the  letters.  For  example, 
if  the  letter  pair  were: 

A A 

Then  you  should  put  a check  mark  to  the  left  of  the  pair 
✓ A A . 

If  the  letter  pair  were: 

A B 

Then  you  should  put  a check  mark  to  the  right  of  the  pair 
A B / . 

You  are  to  work  down  each  column  putting  a check  in  the 
appropriate  place  beside  each  pair  until  you  hear  the  word 
"STOP".  Please  work  as  accurately  and  as  quickly  as  possible. 

Any  questions? 

We  shall  begin  by  giving  you  a few  practice  problems. 


STOP. 

PLEASE  DO  NOT  TURN  THE  PAGE 
UNTIL  INSTRUCTED  TO  DO  SO. 
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RESPONSE  SHEET:  PHYSICAL  MATCH 


Day  2, 

SAME  DIFF 
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STOP 


INSTRUCTIONS  FOR  THE  SENTENCE  RECALL  TASK 


In  this  task  we  are  interested  in  your  ability  to  organize 
and  recall  the  information  presented  in  a list  of  sentences. 

On  any  one  trial  we  are  going  to  present  you  a list  of  sentences 
on  four  topics.  Each  sentence  will  describe  some  aspect  of 
one  of  the  four  topics.  At  the  end  of  the  list  of  sentences 
you  are  to  recall  as  much  as  you  can  about  the  four  topics. 

For  example,  I might  read  you  sentences  like  the  following: 


The  carpenter  fixed  the  roof. 

The  letter  was  typed  by  the  secretary. 
The  shoes  were  in  the  closet. 

The  carpenter  stood  on  a ladder. 

The  shoes  were  made  of  canvas. 

The  carpenter  was  an  apprentice. 

The  letter  was  lengthy. 

The  roof  was  leaking. 

The  shoes  had  wooden  heels. 

The  closet  was  upstairs. 


etc. 

These  sentences  deal  with  the  topics  of  the  carpenter , 
the  shoes,  and  the  letter.  Your  recall  on  the  topic  of  the 
carpenter  might  be  as  follows:  "The  apprentice  carpenter  stood 
on  the  ladder  to  fix  the  leaking  roof."  For  the  shoes  you 
might  write:  "The  canvas  shoes  with  the  wooden  heels  were  in 
the  upstairs  closet."  And  so  on  for  each  of  the  four  topics. 
Your  recall  does  not  have  to  be  complete  sentences;  you  may 
use  pharses  to  express  the  ideas.  For  example,  if  you  wrote 
"apprentice  carpenter  stood  on  ladder,  fixed  leaking  roof" 
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that  would  be  fine.  However,  to  write  "carpenter  ladder 
apprentice  roof  leaking  fixed"  would  not  be  sufficient.  If 
you  use  phrases  make  sure  that  you  give  sufficient  information 
in  the  phrase  to  adequately  express  the  ideas  associated  with 
each  topic.  Try  to  make  sure  you  recall  as  much  as  you  can 
remember  about  the  four  topics  keeping  the  information  about 
each  topic  separate  from  the  other  topics.  You  will  notice 
on  the  response  sheets  that  we  have  provided  four  spaces  for 
you  to  write  your  recall,  one  space  for  each  topic.  You  are 
to  work  down  the  sheet,  recalling  everything  you  can  about 
one  topic  before  going  on  to  the  next. 

So  then,  we  shall  read  you  some  sentences  about  four  topics. 
At  the  end  of  the  list  we  will  say  the  word  recall.  You  will 
then  have  about  two  minutes  in  which  to  recall  all  that  you 
can  about  the  topics.  You  are  to  recall  the  information  by 
topic,  keeping  each  topic  separate. 

Any  questions? 

We  shall  begin  by  giving  you  some  practice. 


STOP. 

PLEASE  DO  NOT  TURN  THE  PAGE 
UNTIL  INSTRUCTED  TO  DO  SO. 
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INSTRUCTIONS  FOR  THE  LETTER  ROTATION  TASK 


In  this  task  we  are  interested  in  your  ability  to  handle  anc 
compare  letters  in  various  orientations.  You  will  see  columns 
of  pairs  of  letters  and  have  to  decide  if  the  letters  in  each 
pair  are  the  same  or  different.  You  are  to  use  two  rules  in 
making  your  decision.  RULE  1 : If  the  two  letters  in  a pair 
are  exactly  the  same  such  that  you  can  simply  slide  the  first 
letter  over  on  top  of  the  second  letter  and  they  match  exactly 
then  put  a check  to  the  left  of  that  pair  (in  the  SAME  column)  . 

For  example,  according  to  Rule  1 the  following  pairs  are  the 
same : 

_/  F F AND  J_  . 

RULE  2 : If  the  first  letter  can  be  rotated  like  the  hands  of 
a clock  so  that  it  will  slide  over  and  exactly  match  the  second 
letter,  then  check  "SAME"  for  this  pair.  For  example,  according 
to  Rule  2 the  following  pairs  are  the  same: 

F 

/ Fi SINCE 

SINCE 

The  following  pairs  however  are  different  since  neither  Rule  1 
nor  Rule  2 will  work. 

Fi,_/  AND  FT  / ■ 

] 

You  cannot  rotate  the  first  letter  like  the  hands  of  a clock 
rotate  to  make  it  match  the  second  letter.  Thus  the  pairs 
should  be  marked  "DIFFERENT." 

For  any  pair  if  Rule  1 or  Rule  2 applies  you  should  mark 
the  pair  as  same  by  putting  a check  to  the  left  of  the  pair. 

If  Rule  1 or  Rule  2 will  not  work  then  you  should  mark  the 
pair  as  different  by  putting  a check  to  the  right  of  the  pair. 

You  should  go  down  the  columns  of  pairs  being  as  accurate  and 
rapid  as  you  can.  When  you  hear  the  word  "STOP"  you  should 
lift  your  pencil  and  wait  for  further  instruction. 
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Any  questions? 

In  the  first  column  of  the  practice  response  sheet  you 
need  only  apply  Rule  1,  that  is,  no  rotation  is  necessary. 

You  should  now  work  through  the  first  column  of  practice 
problems.  Even  though  we  won't  be  timing  you  for  these  prac- 
tice problems  you  should  still  try  to  work  as  fast  as  you  can 
without  making  errors.  Please  look  up  when  you  finish  the 
first  column. 


Any  questions? 

All  the  other  practice  problems  require  that  you  apply 
Rule  2,  to  varying  degrees.  That  is,  you  will  have  to  rotate 
the  first  letter  to  see  if  it  matches  the  second.  You  should 
now  work  the  last  three  columns  on  the  practice  response 
sheet.  Again  try  to  work  as  fast  as  you  can  without  making 
errors.  Please  look  up  when  you  have  done  the  three  columns. 


Any  questions? 

The  first  set  of  problems  in  the  task  will  be  a page  of 
problems  just  like  the  problems  in  column  one.  YOu  will  not 
have  to  rotate  letters.  When  I say  "READY"  you  are  to  turn 
the  page,  when  I say  "START"  you  are  to  work  as  many  of  these 
problems  as  you  can  before  I say  "STOP". 
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RESPONSE  SHEET:  LETTER  ROTATION  TASK  B 
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RESPONSE  SHEET:  LETTER  ROTATION  TASK  D 
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INSTRUCTIONS  FOR  MENTAL  ADDITION  TASK 
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In  the  following  task  we  are  interested  in  your  ability  to 
add  numbers  "in  your  head"  when  the  numbers  you  are  adding  are 
not  written  down.  We  shall  read  two  numbers  which  you  are  to 
add  together.  For  example,  we  might  say:  "Two  hundred  ninety- 
five  plus  four  hundred  fifty-eight."  You  will  have  a fixed 
amount  of  time  in  which  to  do  the  addition.  Try  to  write  down 
as  much  of  the  answer  as  you  can  during  the  time  interval. 

Write  the  sum  down  as  you  get  each  part  of  it.  When  you  hear 
the  word  "STOP",  you  should  stop  working  on  the  problem  and 
get  ready  for  the  next  problem. 


We  are  not  concerned  with  whether  or  not  you  can  complete 
the  problems;  in  fact  for  most  of  the  problems  we  really  don't 
expect  you  to  complete  them.  We  are  most  interested  in  your 
writing  down  the  parts  of  the  answer  as  you  get  them.  On  your 

response  sheets,  there  are  three  blanks  for  each  problem, 

representing  the  units,  tens,  and  hundreds  positions  in  the 
sum.  As  you  get  a part  of  the  sum  write  that  part  in  the 
appropriate  blank.  Do  not  hold  part  of  the  sum  in  memory 

waiting  to  complete  the  addition;  write  out  the  answer  as  you 

get  it.  It  is  important  that  you  be  as  accurate  as  possible 
in  your  sums,  so  please  do  not  guess  about  parts  of  the  answer 
you  have  not  worked  out. 


Any  questions? 


We  shall  begin  by  giving  you  some  practice  problems. 


STOP 
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INSTRUCTIONS  FOR  THE  SCAN  AND  SEARCH  TASK 

In  this  task  we  are  interested  in  your  ability  to  search 
columns  of  letters  looking  for  particular  letters.  At  the 
beginning  of  each  trial  you  will  be  given  a set  of  letters. 

This  set  may  have  1,  2,  or  4 letters.  These  will  be  the  target 
letters  you  will  search  for  on  that  trial.  You  will  work  down 
columns  of  groups  of  letters  looking  for  those  target  letters. 
If  you  find  a group  of  letters  that  contains  a target  letter 
put  a check  beside  that  group.  For  example,  if  the  target 
set  were  "L  or  K"  then  you  should  mark  the  following  as  illus- 
trated : 

Q X M V S 

T R N L 0 / 

K M D A T / 

C P F X R 

P K T S W / 

L A R T D / 

M C N P H 

Notice  that  only  the  groups  containing  an  "L"  or  "K"  are 
checked . 

You  are  to  continue  working  down  the  column  until  you  are 
told  to  "STOP".  When  you  hear  the  word  "STOP"  draw  a line 
under  the  last  group  of  letters  you  looked  at,  not  the  last 
item  you  checked.  At  the  bottom  of  the  page  on  the  right, 
write  the  letters  you  were  searching  for. 

Some  of  the  pages  are  blurred  and  hard  to  read.  This  is 
for  the  purposes  of  the  experiment. 

Any  questions? 

We  shall  begin  by  giving  you  some  practice  on  the  task. 


STOP. 

PLEASE  DO  NOT  TURN  THE  PAGE 
UNTIL  INSTRUCTED  TO  DO  SO. 
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AGBDT 

XJDQE 

AJMPO 

XUMFC 

YZMFU 

AHPRO 

EHQCY 

OTDMB 

UPCJX 

UBMDY 

AFDMX 

AGMQY 

YQMDA 

AFDOG 

RBAET 

OZPDU 

VPROX 

VFCYK 

LQMDG 

JBMDN 

OHFDA 

EQMZU 

XBDMY 

AHPXU 

BMDKL 

AMDBN 

EFDVO 

URVFX 

KQMYL 

PMTNA 

EOUZP 

UYDQE 

YVQCX 

KQDZL 

NBMDA 

AUMFZ 

OBDME 

VPRUX 

KYFMH 

LNDQG 

AEFMZ 

OTMFU 

GPDXY 

KLJDP 

NAPDG 

PDGEO 

UXBMR 

YKTMF 

LBMJN 

BMDKL 

JCQOU 

XVDBY 

YQDTU 

NFCHA 

NAFDT 

ERMPU 

KMDPL 

BDMAE 

EPMZO 

UHQCX 

KHFRY 

NLMQJ 

AEQDV 

OXMDB 

mIS  M*  16  mb  »uum  HU<axo«» 
^O.PIFUBMS"®1001’0 

UMFZ 

y<i'QCt  flV-Zii 

NDQG 

iCVr-MM 

031  Mi  VrXX 

:ljop 

G?!m\ 

Ai-x-r-:  or*-Fu 

KTMF 

fJXBMR 

N.APDG  R03H0 

:vd?  ■ 

wCOOU 

LBM-JN  6M01U 

:*r-vj 

KAFl*F 

VQDTU 

XXX 

CPMZO 

KMDFL  BUMAE 

J A . l.J  -J 

AtQUV 

KHFR'f  ^ 

■ •'* ii  C 

A530T 

XOOQC 

0*  OWl> 

VZMF'J 

AMPRO  f-KJC' 

AGMOV 

UPC.JX 

UBMDY  AFOM> 

INSTRUCTIONS  FOR  THE  SENTENCE  RECOGNITION  TASK 


In  this  task  we  are  interested  in  your  ability  to  recog- 
nize sentences  you  have  heard  before.  We  shall  read  a series 
of  sentences  for  you  to  study  and  remember.  At  the  end  of  the 
series  of  sentences  you  will  be  given  a recognition  test  in 
which  we  shall  again  read  sentences  to  you.  Each  sentence  in 
the  test  will  either  be  a sentence  from  the  series  you  were 
asked  to  study  or  it  will  be  a new  sentence.  You  are  to 
decide  if  each  test  sentence  was  in  the  series  you  studied 
earlier.  The  new  sentences  will  be  very  similar  to  the  sen- 
tences from  the  earlier  series,  so  you  must  study  these  sen- 
tences very  closely.  We  are  asking  for  recognition  of  exact 
wording  as  well  as  events.  If  the  test  sentence  is  exactly 
as  you  saw  it  in  the  series  you  studied  earlier  cross  out  the 
"YES"  on  the  response  sheet;  if  the  test  sentence  is  new  and 
was  not  in  the  earlier  series  then  cross  out  "NO"  on  your 
response  sheet.  In  the  blank  space  beside  your  answer,  you 
are  to  write  a number  from  one  to  five  indicating  how  confident 
you  are  that  your  answer  is  correct.  Write  5 for  very  high 
confidence,  4 for  high  confidence,  3 for  moderate  confidence, 

2 for  low  confidence  and  1 for  very  low  confidence.  Respond 
to  every  sentence  that  you  hear  on  the  test  and  do  not  leave 
any  sentences  blank. 

To  repeat,  you  will  hear  some  sentences  which  you  will 
study  and  try  to  remember  exactly  as  they  are  presented. 

Foil  ing  the  study  sentences  you  will  be  given  a test  and 
have  to  decide  whether  or  not  each  sentence  on  the  test  was 
one  of  the  study  sentences.  Cross  out  the  "YES"  if  it  was, 
and  the  "NO"  if  it  was  not.  Beside  your  answer  indicate  how 
confident  you  are  in  your  answer,  with  5 meaning  very  high 
confidence  and  1 meaning  very  low  confidence. 

Any  questions? 

We  shall  now  give  you  some  practice  trials. 


DAY  2 


NAME 


RESPONSE  SHEET:  SENTENCE  RECOGNITION,  PART  A 


RATINGS: 

1 

very 

2 

low 

3 

moderate 

4 

5 

very  high 

RESPONSE 

RATING 

RESPONSE  RATING 

1.  YES 

NO 

8. 

YES  NO 

2.  YES 

NO 

9. 

YES  NO 

3.  YES 

NO 

10. 

YES  NO 

4,  YES 

NO 

11. 

YES  NO 

5.  YES 

'!0 

12. 

YES  NO 

6.  YES 

NO 

13. 

YES  NO 

7.  YES 

NO 

14. 

YES  NO 
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INSTRUCTIONS  FOR  THE  SET  MEMBERSHIP  TASK 

In  this  task  we  are  interested  in  how  quickly  you  can 
recognize  items  you  have  heard  before.  At  the  start  of  each 
trial  you  will  memorize  a set  of  letters.  You  will  then  work 
down  a column  of  letters  indicating  whether  or  not  each  letter 
was  in  the  set  you  memorized.  If  a particular  letter  was  in 
the  memorized  set  put  a check  to  the  left  of  the  letter.  If 
a letter  was  not  in  the  memorized  set  then  put  a check  to  the 
right  of  the  letter.  For  example,  if  the  memorized  set  was 
"L  T R S W"  then  your  responses  to  the  following  letters  should 
be  as  illustrated: 


You  are  to  work  as  accurately  and  as  quickly  as  possible  until 
you  hear  the  word  "STOP"  at  which  time  you  should  stop  making 
the  checks.  At  the  bottom  right  of  the  page  we  then  want  you 
to  write  down  the  letters  you  memorized. 

Any  questions? 

We  shall  now  give  you  some  practice. 


STOP. 

PLEASE  DO  NOT  TURN  THE  PAGE. 
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APPENDIX  B 


ANALYSIS  OF  VARIANCE  SUMMARY  TABLES 


TABLE  OF  CONTENTS 

Table  No.  Page 

B-l  Anova  of  Letter  Recall:  Order  Recall.  ...  91 

B-2  Anova  of  Letter  Recall:  Derived-Free 

Recall 93 

B-3  Anova  of  Letter  Rotation:  Correct  95 

B-4  Anova  of  Letter  Rotation:  Completed  . . 96 

B-5  Anova  of  Set  Membership:  Completed 97 

B-6  Anova  of  Set  Membership:  Correct 98 

B-7  Anova  of  Sentence  Recall:  Cluster  99 

B-8  Anova  of  Sentence  Recall:  Item 100 

B-9  Anova  of  Scan  and  Search:  Completed  ....  101 

B-10  Anova  of  Scan  and  Search:  Correct 103 

B-l 1 Anova  of  Sentence  Recognition:  Without 

Level  Four 105 

B-12  Anova  of  Mental  Addition:  Correct  108 

B-l 3 Anova  of  Mental  Addition:  Blanks Ill 
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B-l 

ANOVA  OF  LETTER  RECALL:  ORDER  RECALL 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

60762.55 

1 

60762.55 

4353. 11* 

Subjects 

935.22 

67 

13.96 

D (Day) 

164.71 

1 

164.71 

60.33* 

De1 

182.90 

67 

2.73 

S (Administration) 

47.85 

1 

47.85 

25.65* 

Se 

124.99 

67 

1.87 

DS 

23.89 

1 

23.89 

16.31* 

DSe 

98.11 

67 

1.46 

C (No.  of  Letters) 

897.35 

5 

179.47 

80.32* 

Ce 

748.54 

335 

2.23 

DC 

38.05 

5 

7.61 

4.46* 

DCe 

571.68 

335 

1.71 

SC 

49.87 

5 

9.97 

6.21* 

SCe 

537.63 

335 

1.60 

DSC 

23.16 

5 

4.63 

3.17* 

DSCe 

490.18 

335 

1.46 

R (Repetition) 

19.42 

2 

9.71 

8.02* 

Re 

162.14 

134 

1.21 

DR 

23.74 

2 

11.87 

6.29* 

DRe 

252.82 

134 

1.89 

* p < 0.01 

JIn  each  case,  e represents  the  error  tern  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-l 

ANOVA  OF  LETTER  RECALL:  ORDER  RECALL  (Cont'd.) 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

SR 

36.57 

2 

18.29 

13.94* 

SRe1 

175.76 

134 

1.31 

DSR 

19.16 

2 

9.53 

6.67* 

DSRe 

192.50 

134 

1.44 

CR 

76.46 

10 

7.65 

5.10* 

CRe 

1005.32 

670 

1.50 

DCR 

56.21 

10 

5.62 

4.13* 

DCRe 

911.90 

670 

1.36 

SCR 

98.22 

10 

9.82 

6.80* 

SCRe 

98.22 

670 

1.44 

DSCR 

49.48 

10 

4.95 

3.23* 

DSCRe 

1025.53 

670 

1.53 

* p < 0.01 

*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 


i 
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B-2 

ANOVA  OF  LETTER  RECALL:  DERIVED-FREE  RECALL 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

Mean 

84650.39 

1 

84650.39 

12476.41* 

Subjects 

454.58 

67 

6.78 

D (Day) 

27.36 

1 

27.36 

23.71* 

De1 

77.31 

67 

1.15 

S (Administration) 

2.21 

1 

2.21 

2.56 

Se 

57.74 

67 

0.86 

DS 

3.78 

1 

3.78 

4.80 

DSe 

52.78 

67 

0.78 

C (No.  of  Letters) 

390.36 

5 

78.07 

89.82* 

Ce 

291.19 

335 

0.87 

DC 

16.33 

5 

3.27 

5.03* 

DCe 

217.51 

335 

0.65 

SC 

11.71 

5 

2.34 

4.06* 

SCe 

193.18 

335 

0.58 

DSC 

8.2i 

5 

1.64 

2.86 

DSCe 

192.73 

335 

0.58 

R (Repetition) 

13.88 

2 

6.94 

13.21* 

Re 

70.42 

134 

0.53 

DR 

17.25 

2 

8.63 

11.88* 

DRe 

97.33 

134 

0.73 

* p < 0.01 

*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-2 

ANOVA  OF  LETTER  RECALL:  DERIVED-FREE  RECALL  (Cont'd.) 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

SR 

17.97 

2 

8.98 

16.91* 

SRe1 

71.17 

134 

0.53 

DSR 

3.41 

2 

1.71 

2.34 

DSRe 

97.78 

134 

0.73 

CR 

40.15 

10 

4.01 

6.68* 

CRe 

402.55 

670 

0.60 

DCR 

50.30 

10 

5.03 

8.08* 

DCRe 

417.11 

670 

0.62 

SCR 

60.17 

10 

6.02 

10.97* 

SCRe 

367 . 36 

670 

0.55 

★ 

p < 0.01 

1 

In  each  case,  e represents 
and  is  treatment  by  subject 

the  error  term 
interaction. 

of  the  treatment  above 

it 
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ANOVA  OF  LETTER  ROTATION:  CORRECT 


DEGREES  OF 


SOURCE  SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

Mean 

2875.80 

1 

2875.80 

505.40* 

Subjects 

369 . 86 

65 

5.69 

D (Day) 

46.29 

1 

46.29 

78.07* 

De1 

38.54 

65 

0.59 

C (Condition) 

341.37 

3 

113.79 

62.16* 

Ce 

356.97 

195 

1.83 

DC 

4.01 

3 

1.34 

3.07 

DCe 

85.00 

195 

0.44 

*p  < 0.01 

*In  each  case, 
and  is  treatment  by 

e represents  the  error  term 
subject  interaction. 

of  the  treatment 

above  it 

B-4 

ANOVA  OF  LETTER  ROTATION:  COMPLETED 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

2026.83 

1 

2026.83 

866.85* 

Subjects 

151.98 

65 

2.34 

D (Day) 

20.14 

1 

20.14 

77.10* 

De1 

16.98 

65 

0.26 

C (Condition) 

199.83 

3 

66.61 

133.63* 

Ce 

97.20 

195 

0.50 

DC 

0.89 

3 

0.30 

2.01 

DCe 

26.10 

195 

0.15 

*p  < O.Ol 

lIn  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 


B-5 

ANOVA  OF  SET  MEMBERSHIP:  COMPLETED 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

Mean 

602.32 

1 

602.32 

3500.66* 

Subjects 

11.53 

67 

0.17 

D (Day) 

2.55 

1 

2.55 

197.42* 

De1 

0.86 

67 

0.01 

C (Condition) 

7.42 

3 

2.47 

371.68* 

Ce 

1.34 

201 

0.01 

DC 

0.03 

3 

0.01 

3.08** 

DCe 

0.68 

201 

0.003 

k (Repetition) 

0.06 

2 

0.03 

7.38* 

Re 

0.51 

134 

0.004 

DR 

0.07 

2 

0.03 

7.13* 

DRe 

0.63 

134 

0.005 

CR 

0. 16 

6 

0.03 

7.35* 

CRe 

1.44 

402 

0.004 

DCR 

0.20 

6 

0.03 

8.26* 

DCRe 

1.65 

402 

0.004 

* p < 0.01 

**  p < 0.05 

^In  each  case,  e represents  the  error  term  of  the  treatment  above  it  and 
is  treatment  by  subject  interaction. 
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B-6 

ANOVA  OF  SET  MEMBERSHIP:  CORRECT 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

o-6.26 

1 

626 . 26 

2538.56* 

Subjects 

16.53 

67 

0.25 

D (Day) 

1.82 

1 

1.82 

26.90* 

De1 

4.54 

67 

0.07 

C (Condition) 

9.24 

3 

3.08 

50.31* 

Ce 

12.31 

201 

0.06 

DC 

0.28 

3 

0.09 

1.60 

DCe 

11.82 

201 

0.06 

R (Repetition) 

0.27 

2 

0.13 

2.29 

Re 

7.82 

134 

0.06 

DR 

0.06 

2 

0.03 

0.53 

DRe 

7.46 

134 

0.06 

CR 

0.55 

6 

0.09 

1.59 

CRe 

22.97 

402 

0.06 

DCR 

0.54 

6 

0.09 

1.53 

DCRe 

23.54 

402 

0.06 

* p < 0.01 


*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-7 


ANOVA  OF  SENTENCE  RECALL:  CLUSTER 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

806.81 

1 

806.81 

1906.00* 

Subj  ects 

28.36 

67 

0.42 

D (Day) 

0.02 

1 

0.02 

0.23 

De1 

4.78 

67 

0.07 

S (Sentence) 

6.85 

11 

0.62 

2.21* 

A (Attribute) 

1.20 

2 

0.60 

10.04* 

R (Remainder) 

5.65 

9 

0.63 

12.66* 

Se 

37.61 

737 

0.05 

Ae 

8.00 

134 

0.06 

Re 

29.61 

603 

0.05 

DS 

4.51 

11 

G.-41 

7.64* 

DA 

0.18 

2 

0.09 

1.39* 

DR 

4.33 

9 

0.48 

9.33* 

DSe 

39.60 

737 

0.05 

DAe 

8.53 

134 

0.06 

DRe 

31 .07 

603 

0.05 

* p < 0.01 

*In  each  case,  e represents  the  error  term  and  is  the  treatment  by 
subject  interaction. 
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ANOVA  OF  SENTENCE  RECALL:  ITEM 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

Mean 

941.37 

1 

941.37 

3452.03* 

Subjects 

18.27 

67 

0.27 

D (Day) 

0.00 

1 

0.00 

0.00 

De1 

4.78 

67 

0.07 

S (Sentence) 

6.17 

11 

0.56 

14.60* 

A (Attribute) 

0.85 

2 

0.42 

10.27* 

R (Remainder) 

5.32 

9 

0.59 

15.74* 

Se 

28.17 

737 

0.04 

Ae 

5.53 

134 

0.04 

Re 

22.64 

603 

0.04 

DS 

4.52 

11 

0.41 

10.66* 

DA 

0.55 

2 

0.27 

6.16* 

DR 

3.97 

9 

0.44 

11.83* 

DSe 

28.43 

737 

0.04 

DAe 

5.94 

134 

0.04 

DRe 

22.49 

603 

0.34 

* p < 0.01 

^In  each  case,  e represents  the  error  term  of 
and  is  treatment  by  subject  interaction. 

the  treatment,  above 

it 

1 B-9 

ANOVA  OF  SCAN  AND  SEARCH:  COMPLETED 

r 

DEGREES  OF 

SOURCE  SUM  OF  SQUARES  FREEDOM  MEAN  SQUARE  F 


J . Mean 

1245.82 

1 

1245.82 

2585.14* 

Subjects 

32.29 

67 

0.48 

D (Day) 

2.57 

1 

2.57 

68 . 00* 

De1 

2.54 

67 

0.04 

S (Administration) 

n 

0.44 

1 

0.44 

18.85* 

Se 

1.56 

67 

0.02 

DS 

0.34 

1 

0.34 

15.25* 

DSe 

1.48 

67 

0.02 

G (Degraded) 

2.31 

1 

2.31 

83.71* 

Ge 

1.85 

67 

0.03 

DG 

0.003 

1 

0.003 

0.26 

DGe 

0.90 

67 

0.01 

SG 

0.003 

1 

0.003 

0.03 

SGe 

0.77 

67 

0.01 

i - 

DSG 

0.01 

1 

0.01 

0.43 

DSGe 

1.43 

67 

0.02 

C (Condition) 

188.74 

2 

94.37 

634.87* 

•-  Ce 

19.92 

134 

0.15 

DC 

0.87 

2 

0.44 

20.70* 

L 

DCe 

2.82 

134 

0.02 

1.  sc 

0.10 

2 

0.05 

3.06* 

SCe 

2.26 

134 

0.02 

L DSC 

0.03 

2 

0.01 

0.84 

r DSCe 

• *p  < 0.01 

• **p  < 0.05 

2.34 

134 

0.02 

*In  each  case,  e represents  the  error 
treatment  by  subject  interaction. 

term  of  the 

101 

treatment  above 

it  and  is 

ANOVA  OF  SCAN  AND  SEARCH:  COMPLETED  (Cont'd.) 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

GC 

0.12 

2 

0.06 

GCe1 

2.36 

134 

0.02 

DGC 

0.01 

2 

0.005 

DGCe 

1.33 

134 

0.01 

SGC 

0.02 

2 

0.01 

SGCe 

1.60 

134 

0.01 

DSGC 

3.16 

2 

1.58 

DSGCe 

3.15 

134 

0.02 

Pooled  Error 

14.60 

804 

0.02 

F 

3.36* 

0.32 

0.97 

67.18* 


* p < 0.01 

**  p < 0.05 

*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 


I 

I 

I 

C 

[ 

i: 

t 

t 

0 

1: 

i: 

i: 

:: 

1 

i 

i 

i 

i 


B-1C 

ANOVA  OF  SCAN  AND  SEARCH*  CORRECT 


SOURCE  SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

1552.65 

1 

1552.65 

2414.65* 

Subjects 

43.08 

67 

0.64 

D (Day) 

4.37 

1 

4.37 

67.28* 

De1 

4.35 

67 

0.06 

S (Administration) 

0.87 

1 

0.87 

21.28* 

Se 

2.72 

67 

0.04 

DS 

0.47 

1 

0.47 

11.38* 

DSe 

2.75 

67 

0.04 

G (Degraded) 

8.21 

1 

8.21 

150.79* 

Ge 

3.65 

67 

0.05 

DG 

0.16 

1 

0.16 

4.05** 

DGe 

2.62 

67 

0.04 

SG 

0.04 

1 

0.04 

1.77 

SGe 

1.46 

67 

0.02 

DSG 

0.002 

1 

0.002 

0.06 

DSGe 

2.77 

67 

0.04 

C (Condition) 

318.52 

2 

159.76 

741.77* 

Ce 

28,86 

134 

0.22 

DC 

2.17 

2 

1.09 

25.75* 

DCe 

5.66 

134 

0.04 

SC 

0.30 

2 

0.15 

4.50** 

SCe 

4.43 

134 

0.03 

DSC 

0.01 

2 

0.005 

0.08 

DSCe 

4.62 

134 

0.03 

* p < 0.01 

**  p < 0.05 

*In  each  case,  e represents  the  error 
treatment  by  subject  interaction. 

term  of  the 

treatment  above  it 

and  is 
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B-10 

ANOVA  OF  SCAN  AND  SEARCH:  CORRECT  (Cont'd.) 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

GC 

0.82 

2 

0.41 

8.86* 

GCe1 

6.23 

134 

0.05 

DGC 

0.15 

2 

0.08 

2.34 

DGCe 

4.38 

134 

0.03 

SGC 

0.04 

2 

0.02 

0.74 

SGCe 

3.45 

134 

0.03 

DSGC 

4.00 

2 

2.00 

44.44* 

DSGCe 

6.03 

134 

0.05 

Pooled  Error 

28.23 

804 

0.04 

* p < 0.01 

**  p < 0.05 

^In  each  case,  e represents  the  error  term  of  the  treatment  above  it  and 
is  treatment  by  subject  interaction. 
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ANOVA  OF  SENTENCE  RECOGNITION:  WITHOUT  LEVEL  FOUR 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

283.53 

1 

283.53 

11.31* 

Subjects 

1679.18 

67 

25.06 

D (Day) 

389.82 

1 

389.82 

34 . 69* 

De1 

752.88 

67 

11.24 

L (Complexity) 

18.03 

2 

9.32 

0.72 

Le 

1734.78 

134 

12.95 

DL 

206.30 

2 

103.15 

11.46* 

DLe 

1206.11 

134 

9.00 

N (New-Old) 

2527.08 

1 

2527.08 

191.63* 

Ne 

883.55 

67 

13.19 

DN 

875.03 

1 

875.03 

84.71* 

DNe 

692.09 

67 

10.33 

LN 

339.56 

2 

169.78 

18.63* 

LNe 

1221.44 

134 

9.12 

DLN 

586.30 

2 

293.15 

36.13* 

DLNe 

1087.20 

134 

8.11 

I (Inconsistency) 

4626.53 

1 

4626.53 

348.81* 

Ie 

888,68 

67 

13.26 

DI 

517.77 

1 

517.77 

68.36* 

Die 

507.44 

67 

7.57 

LI 

656.68 

2 

328 . 34 

37.82* 

Lie 

1163.49 

134 

8.68 

* p < 0.01 

*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-ll 

ANOVA  OF  SENTENCE  RECOGNITION:  WITHOUT  LEVEL  FOUR  (Cont'd.) 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

DLI 

72.38 

2 

36.19 

3.62** 

1 

DLIe 

1340.28 

134 

10.00 

NI 

32.96 

1 

32.96 

3.54 

NIe 

623.83 

67 

9.31 

DNI 

48.53 

1 

48.53 

6.10** 

DNIe 

533.09 

67 

7.96 

LNI 

237.54 

2 

118.77 

11.44* 

LNIe 

1391.04 

134 

10.38 

DLNI 

447.81 

2 

223.91 

22.25* 

DLNIe 

1348.44 

134 

10.06 

V (Administration) 

240.50 

1 

240.50 

33.42* 

Ve 

482.12 

67 

7.20 

DV 

671.08 

1 

671.08 

80.59* 

DVe 

557.88 

67 

8.33 

LV 

34.72 

2 

17.36 

1.87 

LVe 

1241.28 

134 

9.26 

DLV 

126.22 

2 

63.11 

7.52* 

DLVe 

1124.44 

134 

8.39 

NV 

21.35 

1 

21.35 

2.45 

NVe 

583.52 

67 

8.71 

DNV 

247.06 

1 

247.06 

30.58* 

DNVe 

541.32 

67 

8.08 

* p < 0.01 

**  p < 0.05 

*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-ll 

ANOVA  OF  SENTENCE  RECOGNITION:  WITHOUT  LEVEL  FOUR  (Cont'd.) 

DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

LNV 

446.56 

2 

223.28 

27.88* 

1 

LNVe 

1073.19 

134 

8.01 

DLNV 

207.06 

2 

103.53 

11.25* 

DLNVe 

1233.19 

134 

9.20 

IV 

782.35 

1 

782.35 

72.27* 

IVe 

725.27 

67 

10.82 

DIV 

979.46 

1 

979.46 

136.01* 

DIVe 

482.50 

67 

7.20 

LIV 

217.34 

2 

108.67 

11.29* 

LIVe 

1289.91 

134 

9.63 

DLIV 

145.86 

2 

77.43 

7.07* 

DLIVe 

1468.56 

134 

10.96 

NIV 

29.82 

1 

29.82 

2.68 

NIVe 

744 . 72 

67 

11.16 

DNIV 

519.36 

1 

519.36 

57.01* 

DNIVe 

610.34 

67 

9.11 

LN IV 

347.45 

2 

173.72 

17.62* 

LNIVe 

1321.39 

134 

9.86 

DLNIV 

151.26 

2 

75.63 

7.86* 

DLNIVe 

1289.90 

134 

9.63 

* 

P 

< 0.01 

** 

P 

< 0.05 

1 

In 

each  case,  e represents 

the  error  term  of 

the  treatment  above 

it 

and  is  treatment  by  subject 

interaction. 
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/ ;OVA  OF  MENTAL  ADDITION:  CORRECT 


SOURCE 

SUM  OF  SQUARES 

DEGREES  OF 
FREEDOM 

MEAN  SQUARE 

F 

Mean 

8090.82 

1 

8090.82 

5243.08* 

Subjects 

111.11 

72 

1.54 

D (Day) 

0.49 

1 

0.49 

1.81 

De1 

19.40 

72 

0.27 

S (Administration) 

3.85 

1 

3.85 

28.65* 

Se 

9.67 

72 

0.13 

DS 

0.37 

1 

0.37 

3.54 

DSe 

7.59 

72 

0.11 

L (Long -Short) 

28.18 

1 

28.18 

213.50* 

Le 

9.50 

72 

0.13 

DL 

0.52 

1 

0.52 

6.40** 

DLe 

5.83 

72 

0.08 

SL 

0.02 

1 

0.02 

0.14 

SLe 

8.22 

72 

0.11 

DSL 

0.25 

1 

0.25 

3.19 

DSLe 

5.61 

72 

0.08 

C (Condition) 

64.26 

3 

21.42 

153.33* 

Ce 

30.18 

216 

0.14 

DC 

1.54 

3 

0.51 

6.57* 

DCe 

16.83 

216 

0.08 

SC 

0.98 

3 

0.33 

3.74** 

SCe 

18.89 

216 

0.09 

DSC 

3.00 

3 

1.00 

13.10* 

DSCe 

16.47 

216 

0.08 

* p < 0.01 

**  p < 0.05 

*111  each  case,  e represents  the  error  term  of  the  treatment  above  it 

and  is  treatment  by  subject  interaction. 


B-12 

ANOVA  OF  MENTAL  ADDITION:  CORRECT  (Cont'd.) 

DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

LC 

6.58 

3 

2.19 

24.87* 

LCe 1 

19.03 

216 

0.09 

DLC 

2.65 

3 

0.88 

10.11* 

DLCe 

18.87 

216 

0.09 

SLC 

0.35 

3 

0.12 

1.37 

SLCe 

18.57 

216 

0.09 

DSLC 

3.39 

3 

1.13 

13.34* 

DSLCe 

18.29 

216 

0.08 

R (Repetitions) 

7.25 

2 

3.62 

37.72* 

Re 

13.83 

144 

0.10 

DR 

3.39 

2 

1.69 

17.13* 

DRe 

14.25 

144 

0.10 

SR 

0.55 

2 

0.27 

3.06** 

SRe 

12.63 

144 

0.09 

DSR 

1.32 

2 

0.66 

7.90* 

DSRe 

12.03 

144 

0.08 

LR 

2.34 

2 

1.17 

14.23* 

LRe 

11.82 

144 

0.08 

DLR 

0.69 

2 

0.35 

3.66** 

DLRe 

13.60 

144 

0.09 

* p < 0.01 

**  p < 0.05 

In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-12 


0 


SOURCE 


DSCRe 


DLCRe 


SLCRe 


DSLCR 


DSLCRe 


DSLRe 


* p < 0.01 

**  p < 0.05 


ANOVA  OF  MENTAL  ADDITION:  CORRECT  (Cont'd.) 


SUM  OF  SQUARES 


37.22 


10.15 


37.35 


35.39 


39.69 


36.73 


37.69 


32.87 


39.40 


12.98 


14.53 


DEGREES  OF 
FREEDOM 


MEAN  SQUARE 


19.57* 


5.18* 


13.54* 


18.25* 


In  each  case,  e represents  the  error  term  of  the  treatment  above  it  and 
is  treatment  by  subject  interaction. 
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B-13 

ANOVA  OF  MENTAL  ADDITION:  BLANKS 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

Mean 

814.88 

1 

814.88 

488.28* 

Subjects 

120. 16 

72 

1.67 

D (Day) 

0.98 

1 

0.98 

8.23* 

De1 

8.60 

72 

0.12 

S (Administration) 

1.13 

1 

1.13 

12.91* 

Se 

6.29 

72 

0.09 

DS 

1.40 

1 

1.40 

15.75* 

DSe 

6.39 

72 

0.09 

L (Short -Long) 

21.45 

1 

21.45 

144.10* 

Le 

10.72 

72 

0.15 

C (Condition) 

39.84 

3 

13.28 

119.36* 

Ce 

24.03 

216 

0.11 

DC 

1.42 

3 

0.47 

7.09* 

DCe 

14.44 

216 

0.07 

DSC 

1.37 

3 

0.46 

7.98* 

DSCe 

12.35 

216 

0.06 

LC 

0.63 

3 

0.21 

3.66** 

LCe 

12.45 

216 

0.06 

DLC 

0.66 

3 

0.  22 

3.27** 

DLCe 

14.50 

216 

0.07 

DS  LC 

1.80 

3 

0.60 

10.03* 

DSLCe 

12.92 

216 

0.06 

R (Repetition) 

1.51 

0.76 

10.82* 

Re 

10.05 

144 

0.07 

* p < 0.01 

**  p < 0.05 

In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 
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B-13 

ANOVA  OF  MENTAL  ADDITION:  BLANKS 


DEGREES  OF 


SOURCE 

SUM  OF  SQUARES 

FREEDOM 

MEAN  SQUARE 

F 

Mean 

814.88 

1 

814.88 

488.28* 

Subjects 

120.16 

72 

1.67 

D (Day) 

0.98 

1 

0.98 

8.23* 

De1 

8.60 

72 

0.12 

S (Administration) 

1.13 

1 

1.13 

12.91* 

Se 

6.29 

72 

0.09 

DS 

1.40 

1 

1.40 

15.75* 

DSe 

6.39 

72 

0.09 

L (Short -Long) 

21.45 

1 

21.45 

144.10* 

Le 

10.72 

72 

0.15 

C (Condition) 

39.84 

3 

13.28 

119.36* 

Ce 

24.03 

216 

0.11 

DC 

1.42 

3 

0.47 

7.09* 

DCe 

14.44 

216 

0.07 

DSC 

1.37 

3 

0.46 

7.98* 

DSCe 

12.35 

216 

0.06 

LC 

0.63 

3 

0.21 

3.66** 

LCe 

12.45 

216 

0.06 

DLC 

0.66 

3 

0.22 

3.27** 

DLCe 

14.50 

216 

0.07 

DS  LC 

1.80 

3 

0.60 

10.03* 

DSLCe 

12.92 

216 

0.06 

R (Repetition) 

1.51 

2 

0.76 

10.82* 

Re 

10.05 

144 

0.07 

* p < 0.01 

**  p < 0.05 

*In  each  case,  e represents  the  error  term  of  the  treatment  above  it 

and  is  treatment  by  subject  interaction. 


Ill 


B-13 

ANOVA  OF  MENTAL  ADDITION:  BUNKS  (Cont'd.) 


SOURCE 


SUM  OF  SQUARES 


27.46 


28.44 


26.68 


DEGREES  OF 
FREEDOM 


MEAN  SQUARE 


20.96* 


5.42* 


DSCRe 


24.76 


26.75 


DLCRe 


29.12 


SLCRe 


23.20 


DSLCR 


DSLCRe 


26.21 


* p < 0.01 


In  each  case,  e represents  the  error  term  of  the  treatment  above  it 
and  is  treatment  by  subject  interaction. 


112 


i: 

I o 

! fi 

I I 
; C 

E 0 

I 

I 


Distribution  List 


NAVY 

Dr . Ed  Aiken 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

Dr.  Jack  R.  Borsting 
Provost  & Academic  Dean 
U.S.  Naval  Postgraduate  School 
Monterey,  CA  93940 

Dr.  Robert  Breaux 
Code  N-71 
NAVTRAEQUIPCEN 
Orlando,  FL  32813 

Mr.  Maurice  Callahan 
Pers  23a 

Bureau  of  Naval  Personnel 
Washington,  DC  20370 

Chief  of  Naval  Education  and 
Training  Support  - (01A) 
Pensacola,  FL  32509 

Dr.  Richard  Elster 
Department  of  Administrative 
Sciences 

Naval  Postgraduate  School 
Monterey,  CA  93940 

Dr.  Pat  Federico 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152 

CDR  John  Ferguson,  MSC,  USN 
Naval  Medical  R&D  Command 
Code  4 4 

National  Naval  Medical  Center 
Bethesda,  MD  20014 

Dr.  Richard  Gibson 

Bureau  of  Medecine  and  Surgery 

Code  513 

Navy  Department 

Washington,  DC  20372 

Capt.  D.M.  Gragg,  MC,  USN 
Head,  Section  on  Medical  Education 
Uniformed  Services  University  of 
the  Health  Sciences 
6917  Arlington  Road 
Bethesda,  MD  20014 


Mr.  George  N.  Graine 
Naval  Sea  Systems  Command 
Sea  047C112 
Washington,  DC  20362 

Dr.  Steve  Harris 
Code  L522 
MANRL 

Pensacola,  FL  32508 

LCDR  Charles  W.  Hutchins 
Naval  Air  Systems  Command 
444  Jefferson  Plaza  #1 
1411  Jefferson  Davis  Highway 
Arlington,  VA  20360 

CDR  Robert  S.  Kennedy 
Naval  Aerospace  Medical  and 
Research  Lab 
Box  29407 

New  Orleans,  TN  38054 

Dr.  Norman  J.  Kerr 
Chief  of  Naval  Technical 
Training 

Naval  Air  Station  Memphis  (75) 
Millington,  TN  38054 

Dr.  Leonard  Kroeker 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

Chairman,  Leadership  & Law 
Department 

Division  of  Professional 
Development 
U.S.  Naval  Academy 
Annapolis,  MD  21402 

Dr.  William  L.  Maloy 
Principal  Civilian  Advisor  for 
Education  and  Training 
Naval  Training  Command 
Code  00A 

Pensacola,  FL  32508 

Capt.  Richard  L.  Martin 
USS  Francis  Marion  (LPA-Z49) 
FPO  New  York,  NY  09501 


% 


1 Dr.  James  McBride 
Code  301 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1 Dr.  William  Montague 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1 Commanding  Officer 

U.S.  Naval  Amphibious  School 
Coronado,  CA  92155 

1 Commanding  Officer 

Naval  Health  Research  Center 

Attn:  Library 

San  Diego,  CA  92152 

1 Naval  Medical  R&D  Command 
Code  44 

National  Naval  Medical  Center 
Bethesda,  MD  20014 

1 Capt.  Paul  Nelson,  USN 

Chief,  Medical  Service  Corps 
Code  7 

Bureau  of  Medicine  & Surgery 
U.S.  Department  of  the  Navy 
Washington,  DC  20372 

1 Library 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

6 Commanding  Officer 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20390 

1 Office  of  Civilian  Personnel 
Code  26 

Department  of  the  Navy 
Washington,  DC  20390 

1 John  Olsen 

Chief  of  Naval  Education  & 
Training  Support 
Pensacola,  FL  32509 


1 Psychologist 

ONR  Branch  Office 
495  Summer  Street 
Boston,  MA  02210 

1 Psychologist 

ONR  Branch  Office 
536  S.  Clark  Street 
Chicago,  IL  60605 

1 Office  of  Naval  Research 
Code  200 

Arlington,  VA  22217 

1 Office  of  Naval  Research 
Code  441 

800  N.  Quincy  Street 
Arlington,  VA  22217 

5 Personnel  & Training  Research 
Programs 
Code  458 

Office  of  Naval  Research 
Arlington,  VA  22217 

1 Psychologist 

Office  of  Naval  Research  Branch 
223  Old  Marylebone  Road 
London,  NW,  15th  England 

1 Psychologist 

ONR  Branch  Office 
1030  East  Green  Street 
Pasadena,  CA  91101 

1 Scientific  Director 

Office  of  Naval  Research 
Scientific  Liaison  Group/Tokyo 
American  Embassy 
APO  San  Francisco,  CA  96503 

1 Head,  Research,  Development  and 
Studies  (OP102X) 

Office  of  the  Chief  of  Naval 
Operations 

Washington,  DC  20370 

1 Scientific  Advisor  to  the  Chief 
of  Naval  Personnel  (Pers-Or) 
Naval  Bureau  of  Personnel 
Room  4410,  Arlington  Annex 
Washington,  DC  20370 


1 Dr.  Richard  A.  Poliak 

Academic  Computing  Center 
U.S.  Naval  Academy 
Annapolis,  MD  21402 

1 Mr.  Arnold  Rubenstein 

Naval  Personnel  Support  Technology 
Naval  Material  Command  (08T244) 
Room  1044,  Crystal  Plaza  #5 
2221  Jefferson  Davis  Highway 
Arlington,  VA  20360 

1 Dr.  Worth  Scanland 

Chief  of  Naval  Education  and 
Training 
Code  N-5 

NAS,  Pensacola,  FL  32508 

1 A. A.  Scjoholm 

Technical  Support 
Code  201 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1 Mr.  Robert  Smith 

Office  of  Chief  of  Naval 
Operations 
OP-987E 

Washington,  DC  20350 

1 Dr.  Alfred  F.  Smode 

Training  Analysis  & Evaluation 
Group 
(TAEG) 

Department  of  the  Navy 
Orlando,  FL  32813 

1 CDR  Charles  J.  Theisen,  Jr.  M SC, 
USN 

Head  Human  Factors  Engineering 
Division 

Naval  Air  Development  Center 
Warminster,  PA  18974 

1 W.  Gary  Thomson 

Naval  Ocean  Systems  Center 
Code  7132 

San  Diego,  CA  92152 

1 Dr.  H.M.  West  III 

Deputy  ADCNO  for  Civilian 
Planning  and  Programming 
Room  2625,  Arlington  Annex 
Washington,  DC  20370 


1 Dr.  Martin  F.  Wiskoff 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

ARMY 

1 ARI  Field  Unit-USAREUR 
Attn:  Library 
c/o  ODCSPER 
HQ  USAREUR  & 7th  Army 
APO  New  York  09403 

1 Technical  Director 

U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social 
Sciences 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1 HQ  USAREUE  & 7th  Army 
ODCSOPS 

USAAREUE  Director  of  GED 
APO  New  York  09403 

1 Dr.  Ralph  Canter 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1 Dr.  Milton  S.  Katz 

Individual  Training  & Skill 
Evaluation  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1 Dr.  Harold  F.  O'Neil,  Jr. 

Attn:  PERI-OK 

5001  Eisenhower  Avenue 

Alexandria,  VA  22333 

1 Dr.  James  L.  Raney 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1 Director,  Training  Development 
U.S.  Army  Administration  Center 
Attn:  Dr.  Sherrill 
Ft.  Benjamin  Harrison,  IN  46218 

1 Dr.  Joseph  Ward 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 


I 


t OTWIMBwr. 


i i: 


AIR  FORCE 

1 Air  Force  Human  Resources  Lab 
AFHRL/PED 

Brooks  AFB,  TX  78235 

1 Air  University  Library 
AUL/LSE  76/443 
Maxwell  AFB,  AL  36112 

1 CDR  Mercer 

CNET  Liaison  Officer 
AFHRL/Flying  Training  Division 
Williams  AFB,  AZ  85224 

1 Personnel  Analysis  Division 
HQ  USAF/DPXXA 
Washington,  DC  20330 

1 Research  Branch 
AFMPC/DPMYP 
Randolph  AFB,  TX  78148 

1 Dr.  Marty  Rockway  AFHRL/TT 
Lowry  AFB 
Colorado  80230 

1 Jack  A.  Thorpe,  Capt.,  USAF 
Program  Manager 
Life  Sciences  Directorate 
AFOSR 

Bolling  AFB,  DC  20332 

1 Brian  K.  Waters,  LCOL,  USAF 
Air  University 
Maxwell  AFB 
Montgomery,  AL  36112 

MARINES 

1 Director,  Office  of  Manpower 
Utilization 
HQ  Marine  Corps  (MPU) 

BCB,  Building  2009 
Quantico,  VA  22134 

1 MCDEC 

Quantico  Marine  Corps  Base 
Quantico,  VA  22134 

1 Dr.  A. L.  Slafkosky 
Scientific  Advisor 
Code  RD-1 

HQ  U.S.  Marine  Corps 
Washington,  DC  20380 


COAST  GUARD 

1 Mr.  Joseph  J.  Cowan,  Chief 

Psychological  Research  (G-P-l/62) 
U.S.  Coast  Guard  HQ 
Washington,  DC  20590 

OTHER  DOD 

1 Dr.  Stephen  Andriole 

Advanced  Research  Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

12  Defense  Documentation  Center 
Cameron  Station,  Building  5 
Alexandria,  VA  22314 
Attn:  TC 

1 Dr.  Dexter  Fletcher 

Advanced  Research  Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

1 Military  Assistant  for  Training 
and  Personnel  Technology 
Office  of  the  Under  Secretary 
of  Defense  for  Research  and 
Engineering 

Room  3D129,  The  Pentagon 
Washington,  DC  20301 

1 Major  Wayne  Sellman,  USAF 

Office  of  the  Assistant  Secretary 
of  Defense  (MRA&L) 

Room  3B930,  The  Pentagon 
Washington, DC  20301 

1 Mr.  Fredrick  W.  Suffa 
MPP  (A&R) 

Room  2B269,  The  Pentagon 
Washington,  DC  20301 

CIVIL  GOVT. 

1 Dr.  Susan  Chipman 
Basic  Skills  Program 
National  Institute  of  Education 
1200  19th  Street  N.W. 

Washington,  DC  20208 

1 Dr.  Lorraine  D.  Eyde 
Personnel  R&D  Center 
U.S.  Civil  Service  Commission 
1900  E Street  N.W. 

Washington,  DC  20415 


J 


► 


* 


i 


i 

I 

i 

i 

t 

L 

I 


i 

i: 

t 

i 

i 

i 

i 

i 


1 Dr.  William  Gorham,  Director 
Personnel  R&D  Center 
U.S.  Civil  Service  Commission 
1900  E Street  N.W. 

Washington,  DC  20415 

1 Dr.  Joseph  I.  Lipson 

Division  of  Science  Education 
Room  W-638 

National  Science  Foundation 
Washington,  DC  20550 

1 Dr.  John  Mays 

National  Institute  of  Education 
1200  19th  Street  N.W. 
Washington,  DC  20208 

1 National  Institute  of  Education 
1200  19th  Street  N.W. 
Washington,  DC  20208 

1 Dr.  Andrew  R.  Molnar 

Science  Education  Development 
and  Research 

National  Science  Foundation 
Washington,  DC  20550 

1 Dr.  H.  Wallace  Sinaiko 
Program  Director 
Manpower  Research  and  Advisory 
Services 

Smithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

1 Dr.  Thomas  G.  Sticht 
Basic  Skills  Program 
National  Institute  of  Education 
1200  19th  Street  N.W. 
Washington,  DC  20208 

1 Robert  W.  Stump 

Eeucation  and  Work  Group 
National  Institute  of  Education 
1200  19th  Street  N.W. 
Washington,  DC  20208 

1 Dr.  Vern  W.  Urry 

Personnel  R&D  Center 

U.S.  Civil  Service  Commission 

1900  E Street  N.W. 

Washington,  DC  20415 


1 C.S.  Winiewicz 

U.S.  Civil  Service  Commission 
Regional  Psychologist 
230  S.  Dearborn  Street 
Chicago,  IL  60604 

1 Dr.  Joseph  L.  Young,  Director 
Memory  & Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 

NON  GOVT. 

1 Prof.  Earl  A.  Alluisi 
Department  of  Psychology 
Code  287 

Old  Dominion  University 
Norfolk,  VA  23508 

1 1 Psychological  Research  Unit 

Department  of  Defense  (Army 
Office) 

Campbell  Park  Offices 
Canberra  ACT  2600,  Australia 

1 Dr.  Gerald  V.  Barrett 
Department  of  Psychology 
University  of  Akron 
Akron,  OH  44325 

1 Dr.  Nicholas  A.  Bond 

Department  of  Psychology 
Sacramento  State  College 
600  Jay  Street 
Sacramento,  CA  95819 

1 Dr.  David  G.  Bowers 

Institute  for  Social  Research 
University  of  Michigan 
Ann  Arbor,  MI  48106 

1 Dr.  C.  Victor  Bunderson 
WICAT  Inc. 

University  Plaza,  Suite  10 
1160  S.  State  Street 
Orem,  UT  84057 

1 Dr.  John  B.  Carroll 
Psychometric  Lab 
University  of  North  Carolina 
Davie  Hall  013A 
Chapel  Hill,  NC  27514 


■r* 


Charles  Myers  Library 
Livingstone  House 
Livingstone  Road 
Stratford 

London  E15  2LJ  England 

Dr.  William  Chase 
Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

Dr.  Kenneth  E.  Clark 
College  of  Arts  & Sciences 
University  of  Rochester 
River  Campus  Station 
Rochester,  NY  14627 

Dr.  Norman  Cliff 

Department  of  Psychology 

University  of  Southern  California 

University  Park 

Los  Angeles,  CA  90007 

Dr.  Meredith  Crawford 
Department  of  Engineering 
Administration 
George  Washington  University 
Suite  805 
2101  L Street  N.W. 

Washington,  DC  20037 

Dr . Ruth  Day 

Center  for  Advanced  Study  in 
Behavioral  Sciences 
202  Junipero  Serra  Boulevard 
Stanford,  CA  94305 

Dr.  Marvin  D.  Dunnette 
N492  Elliott  Hall 
Department  of  Psychology 
University  of  Minnesota 
Minneapolis,  MN  55455 

Major  I.N.  Evonic 
Canadian  Forces  Personnel 
Applied  Research 
1107  Avenue  Road 
Toronto,  Ontario,  Canada 

Dr.  Richard  L.  Ferguson 
The  American  College  Testing 
Program 
P.  O.  Box  168 
Iowa  City,  IA  52240 


1 Dr.  Victor  Fields 

Department  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

1 Dr.  Edwin  A.  Fleishman 

Advanced  Research  Resources 
Organization 
8555  Sixteenth  Street 
Silver  Spring,  MD  20910 

1 Dr.  John  R.  Frederiksen 
Bolt,  Beranek  & Newman 
50  Moulton  Street 
Cambridge,  MA  02138 

1 Dr.  Vernon  S.  Gerlach 
College  of  Education 
146  Payne  Building  B 
Arizona  State  University 
Tempe,  AZ  85281 

1 Dr.  Robert  Glaser 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1 Dr.  James  G.  Greeno 
LRDC 

University  of  Pittsburgh 
3939  ) 'Hara  Street 
Pittsburgh,  PA  15213 

1 Dr.  Ron  Hambleton 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01002 

2 Dr.  Barbara  Hayes-Roth 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90406 

1 Dr.  James  R.  Hoffman 

Department  of  Psychology 
University  of  Delaware 
Newark,  DE  19711 

1 Library 

HumRRO/Western  Division 
27857  Berwick  Drive 
Carmel,  CA  93921 


u 


Dr.  Earl  Hunt 
Department  of  Psychology 
University  of  Washington 
Seattle,  WA  98105 

Mr . Gary  Irving 
Data  Sciences  Division 
Technology  Services  Corporation 
2811  Wilshire  Boulevard 
Santa  Monica,  CA  90403 

Dr.  Lawrence  B.  Johnson 
Lawrence  Johnson  & Assoc. , Inc. 
Suite  502 
2001  S Street  N.W. 

Washington,  DC  20009 

Dr.  Roger  A.  Kaufman 
203  Dodd  Hall 
Florida  State  University 
Tallahassee,  FL  32306 

Dr.  Steven  W.  Keele 
Department  of  Psychology 
University  of  Oregon 
Eugene,  OR  97403 

LCOL  C.R.J.  LaFleur 
Personnel  Applied  Research 
National  Defense  Headquarters 
101  Colonel  By  Drive 
Ottawa,  Canada  K1A0K2 

Dr . Alan  Lesgold 
Learning  R&D  Center 
University  of  Pittsburgh 
Pittsburgh,  PA  15260 

Dr.  Frederick  M.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08540 

Dr.  Robert  R.  Mackie 
Human  Factors  Research,  Inc. 

6780  Cortona  Drive 
Santa  Barbara  Research  Pk. 

Goleta,  CA  93017 

Richard  T.  Mowday 

College  of  Business  Administration 
University  of  Oregon 
Eugene,  OR  97403 


1 Dr.  Allen  Munro 

University  of  Southern  California 
Behavioral  Technology  Labs 
3717  South  Hope  Street 
Los  Angeles,  CA  90007 

1 Dr.  Donald  A.  Norman 

Department  of  Psychology  C-009 
University  of  California,  San 
Diego 

La  Jolla,  CA  92093 

1 Dr.  Melvin  R.  Novick 
Iowa  Testing  Programs 
University  of  Iowa 
Iowa  City,  IA  52242 

1 Dr.  Jesse  Orlansky 

Institute  for  Defense  Analysis 
400  Army  Navy  Drive 
Arlington,  VA  22202 

1 Dr.  James  A.  Paulson 

Portland  State  University 
P.  O.  Box  751 
Portland,  OR  97207 

1 Mr.  A.J.  Pesch,  President 
Eclectech  Associates,  Inc. 

P.  O.  Box  178 
N.  Stonington,  CT  06359 

1 Mr.  Luigi  Petrullo 

2431  N.  Edgewood  Street 
Arlington,  VA  22207 

1 Dr.  Steven  M.  Pine 
4950  Douglas  Avenue 
Golden  Valley,  MN  55416 

1 Dr.  Frank  Pratzner 

Center  for  Vocational  Education 
Ohio  State  University 
1960  Kenny  Road 
Columbus,  OH  43210 

1 Dr.  Diane  M.  Ramsey-Klee 

R-K  Research  and  System  Division 
3947  Ridgemont  Drive 
Malibu,  CA  90265 


L Min.  Ret.  M.  Rauch 
P II  4 

Bundesministerium  Der 
Verteigdigung 
Postfach  161 
53  Bonn  1,  Germany 

L Dr.  Peter  B.  Read 

Social  Science  Research  Council 
605  Third  Avenue 
New  York,  NY  10016 

L Dr.  Mark  D.  Reckase 

Educational  Psychology  Department 
University  of  Missouri-Columbia 
12  Hill  Hall 
Columbia,  MO  65201 

L Dr.  Leonard  L.  Rosenbaum,  Chairman 
Department  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

L Dr.  Ernst  Z.  Rothkopf 
Bell  Laboratories 
600  Mountain  Avenue 
Murray  Hill,  NJ  07974 

L Prof.  Fumiko  Samejima 
Department  of  Psychology 
University  of  Tennessee 
Knoxville,  TN  37916 

L Dr.  Benjamin  Schneider 
Department  of  Psychology 
University  of  Maryland 
College  Park,  MD  20742 

L Dr.  Walter  Schneider 
Department  of  Psychology 
University  of  Illinois 
Champaign,  IL  61820 

1 Dr.  Richard  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

L Dr.  Robert  Sternberg 
Department  of  Psychology 
Yale  University 
Box  11A,  Yale  Station 
New  Haven,  CT  06520 


Mr.  D.J.  Sullivan 

c/o  Canyon  Research  Group, Inc. 

741  Lake fie Id  Road 
Westlake  Village,  CA  91361 

Dr.  Patrick  Suppes 
Institute  for  Mathematical 

Studies  in  the  Social  Sciences 
Stanford  University 
Stanford,  CA  94305 

Dr.  Hariharan  Swaminathan 
Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01003 

Dr.  Kikumi  Tatsuoka 
Computer  Based  Education  Research 
Laboratory 

252  Engineering  Research 
Laboratory 

University  of  Illinois 
Urbana,  IL  61801 

Dr.  Perry  Thorndyke 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90406 

Dr.  Benton  J.  Underwood 
Department  of  Psychology 
Northwestern  University 
Evanston,  IL  60201 

Dr.  Thomas  Wallsten 
Psychometric  Laboratory 
Davie  Hall  013 A 
University  of  North  Carolina 
Chapel  Hill,  NC  27514 

Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455 

Dr.  Susan  E.  Whitely 
Psychology  Department 
University  of  Kansas 
Lawrence,  KS  66044 


